2024-04-02T16:11:16.712 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T16:11:16.713 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635705 --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635705/orig.config.yaml 2024-04-02T16:11:16.760 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T16:11:16.849 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635705 branch: wip-batrick-testing-20240402.004512 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} email: pdonnell@redhat.com first_in_suite: false job_id: '7635705' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240402.004512 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240402.004512 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2763 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 sleep_before_teardown: 0 subset: 318/512 suite: fs suite_branch: wip-batrick-testing-20240402.004512 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNbSUWgiMdsDSYGPx6UsoJ1GP0ucxTqSTLsMFQwRZa1BJ31hojCBoDUJx6g0VweD55pl+tav2SeHhVtPxYFlaYw= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFcNDB6+8zz3+sp8lJHB7E/fUtr0BSP2sNheFVRQ/v+J/YulZRb0hF57mceu+lJ4IqwuCl+jC+jOynvD7bdbGfY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:52:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T16:11:16.850 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa; will attempt to use it 2024-04-02T16:11:16.850 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks 2024-04-02T16:11:16.851 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T16:11:16.852 INFO:teuthology.task.internal:Checking packages... 2024-04-02T16:11:16.877 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '191d963ac9de7322da8aff20b567b3fa458e2ef6' 2024-04-02T16:11:16.877 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T16:11:16.877 INFO:teuthology.packaging:ref: None 2024-04-02T16:11:16.877 INFO:teuthology.packaging:tag: None 2024-04-02T16:11:16.877 INFO:teuthology.packaging:branch: wip-batrick-testing-20240402.004512 2024-04-02T16:11:16.877 INFO:teuthology.packaging:sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:11:16.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240402.004512 2024-04-02T16:11:17.018 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2609.g191d963a 2024-04-02T16:11:17.019 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T16:11:17.029 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T16:11:17.029 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T16:11:17.168 INFO:teuthology.task.internal:Saving configuration 2024-04-02T16:11:17.186 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T16:11:17.228 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T16:11:17.250 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635705', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:06:26.197061', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNbSUWgiMdsDSYGPx6UsoJ1GP0ucxTqSTLsMFQwRZa1BJ31hojCBoDUJx6g0VweD55pl+tav2SeHhVtPxYFlaYw='} 2024-04-02T16:11:17.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635705', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 16:06:26.195746', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFcNDB6+8zz3+sp8lJHB7E/fUtr0BSP2sNheFVRQ/v+J/YulZRb0hF57mceu+lJ4IqwuCl+jC+jOynvD7bdbGfY='} 2024-04-02T16:11:17.271 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T16:11:17.282 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-02T16:11:17.282 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-02T16:11:17.282 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T16:11:17.360 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8181e92160>, signals=[15]) 2024-04-02T16:11:17.360 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T16:11:17.371 INFO:teuthology.task.internal:Opening connections... 2024-04-02T16:11:17.371 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:11:17.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:11:17.482 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:11:17.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:11:17.573 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T16:11:17.584 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-04-02T16:11:17.602 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-04-02T16:11:17.602 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-04-02T16:11:17.660 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-04-02T16:11:17.661 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T16:11:17.661 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-04-02T16:11:17.661 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T16:11:17.661 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T16:11:17.661 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T16:11:17.661 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-04-02T16:11:17.703 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-04-02T16:11:17.721 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-04-02T16:11:17.721 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-04-02T16:11:17.778 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-04-02T16:11:17.778 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T16:11:17.779 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T16:11:17.779 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-04-02T16:11:17.803 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T16:11:17.818 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T16:11:17.887 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T16:11:17.887 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-04-02T16:11:17.890 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-04-02T16:11:17.906 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T16:11:17.918 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T16:11:17.918 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-04-02T16:11:17.947 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-04-02T16:11:17.974 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T16:11:18.029 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T16:11:18.051 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-02T16:11:18.051 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-02T16:11:18.051 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-02T16:11:18.051 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T16:11:18.052 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T16:11:18.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:11:18.068 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-04-02T16:11:18.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:11:18.070 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-04-02T16:11:18.124 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-361.el9.x86_64 2024-04-02T16:11:18.125 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.14.0-361.el9.x86_64 2024-04-02T16:11:18.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-04-02T16:11:18.127 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-361.el9.x86_64 2024-04-02T16:11:18.127 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.14.0-361.el9.x86_64 2024-04-02T16:11:18.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2024-04-02T16:11:21.616 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 3.6 MB/s | 6.7 MB 00:01 2024-04-02T16:11:22.060 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 3.0 MB/s | 6.7 MB 00:02 2024-04-02T16:11:24.650 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 1.1 MB/s | 2.0 MB 00:01 2024-04-02T16:11:25.027 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 1.1 MB/s | 2.0 MB 00:01 2024-04-02T16:11:25.856 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-04-02T16:11:27.691 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-02T16:11:31.767 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-04-02T16:11:33.427 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-02T16:11:34.365 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 914 kB/s | 8.0 MB 00:09 2024-04-02T16:11:34.536 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-02T16:11:38.315 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-04-02T16:11:39.627 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-02T16:11:41.960 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T16:11:41.960 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T16:11:42.065 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-04-02T16:11:42.071 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T16:11:42.071 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-04-02T16:11:42.071 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T16:11:42.071 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-04-02T16:11:42.071 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2024-04-02T16:11:42.072 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:11:42.073 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 76 M 2024-04-02T16:11:42.073 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 123 M 2024-04-02T16:11:42.073 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-04-02T16:11:42.296 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 862 kB/s | 192 kB 00:00 2024-04-02T16:11:42.679 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 25 MB/s | 15 MB 00:00 2024-04-02T16:11:43.462 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): kernel-modules-core-5.14.0-434.el9.x86_6 24 MB/s | 28 MB 00:01 2024-04-02T16:11:44.046 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 17 MB/s | 34 MB 00:01 2024-04-02T16:11:44.049 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:11:44.049 INFO:teuthology.orchestra.run.smithi050.stdout:Total 39 MB/s | 76 MB 00:01 2024-04-02T16:11:44.050 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-04-02T16:11:44.476 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-04-02T16:11:44.476 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-04-02T16:11:45.361 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-04-02T16:11:45.361 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-04-02T16:11:47.787 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-04-02T16:11:48.402 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T16:11:48.489 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T16:11:50.289 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 746 kB/s | 6.1 MB 00:08 2024-04-02T16:11:51.368 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T16:11:51.502 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T16:11:51.698 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-02T16:11:52.985 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-02T16:11:56.567 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T16:11:56.609 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:11:57.974 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-02T16:12:00.240 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T16:12:00.240 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T16:12:00.328 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T16:12:00.339 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T16:12:00.340 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T16:12:00.340 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:12:00.340 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-02T16:12:00.340 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-02T16:12:00.340 INFO:teuthology.orchestra.run.smithi188.stdout:Install 4 Packages 2024-04-02T16:12:00.340 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:12:00.340 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 76 M 2024-04-02T16:12:00.341 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 123 M 2024-04-02T16:12:00.341 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-04-02T16:12:01.200 INFO:teuthology.orchestra.run.smithi188.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 223 kB/s | 192 kB 00:00 2024-04-02T16:12:01.716 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:03.333 INFO:teuthology.orchestra.run.smithi188.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 5.0 MB/s | 15 MB 00:02 2024-04-02T16:12:04.325 INFO:teuthology.orchestra.run.smithi188.stdout:(3/4): kernel-modules-core-5.14.0-434.el9.x86_6 8.8 MB/s | 28 MB 00:03 2024-04-02T16:12:04.825 INFO:teuthology.orchestra.run.smithi188.stdout:(4/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 7.5 MB/s | 34 MB 00:04 2024-04-02T16:12:04.828 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:12:04.828 INFO:teuthology.orchestra.run.smithi188.stdout:Total 17 MB/s | 76 MB 00:04 2024-04-02T16:12:04.828 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-04-02T16:12:05.201 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-04-02T16:12:05.201 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-04-02T16:12:06.053 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-04-02T16:12:06.053 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-04-02T16:12:08.352 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-04-02T16:12:08.975 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T16:12:09.068 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T16:12:11.814 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T16:12:11.965 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T16:12:16.682 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T16:12:16.779 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:21.507 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:32.832 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:32.832 INFO:teuthology.orchestra.run.smithi050.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T16:12:32.832 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:12:32.876 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:33.299 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:33.299 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T16:12:33.299 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T16:12:33.299 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T16:12:33.682 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:12:33.683 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-04-02T16:12:34.051 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T16:12:34.161 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T16:12:34.161 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T16:12:34.161 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T16:12:34.162 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T16:12:34.162 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T16:12:34.162 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-02T16:12:34.162 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T16:12:34.162 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T16:12:34.246 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T16:12:34.246 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T16:12:34.246 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 68.el9 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8264254 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T16:12:34.284 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:12:34.285 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T16:12:34.286 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T16:12:34.286 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 5.14.0-434.el9.x86_64 2024-04-02T16:12:34.286 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T16:12:34.874 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-04-02T16:12:36.337 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T16:12:36.380 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-04-02T16:12:36.382 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-04-02T16:12:36.397 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.OPauNnZtya 2024-04-02T16:12:36.397 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OPauNnZtya 2024-04-02T16:12:36.468 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.OPauNnZtya 2024-04-02T16:12:36.614 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.OPauNnZtya is 6KB 2024-04-02T16:12:36.663 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.OPauNnZtya 2024-04-02T16:12:36.677 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T16:12:36.746 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T16:12:36.746 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T16:12:36.746 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T16:12:36.747 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T16:12:36.748 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T16:12:36.901 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-04-02T16:12:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:50.618 INFO:teuthology.orchestra.run.smithi188.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T16:12:50.618 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:12:50.661 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T16:12:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T16:12:51.076 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T16:12:51.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T16:12:51.480 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:12:51.481 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-04-02T16:12:51.481 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T16:12:51.481 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T16:12:51.481 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T16:12:51.481 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T16:12:51.481 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:12:51.481 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-02T16:12:51.801 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T16:12:51.912 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T16:12:51.912 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T16:12:51.912 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T16:12:51.913 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T16:12:51.913 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T16:12:51.913 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-04-02T16:12:51.913 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T16:12:51.913 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T16:12:51.992 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T16:12:51.993 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T16:12:51.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 68.el9 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8264254 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2024-04-02T16:12:52.026 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:12:52.027 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T16:12:52.028 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T16:12:52.028 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 5.14.0-434.el9.x86_64 2024-04-02T16:12:52.029 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T16:12:52.623 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-04-02T16:12:53.976 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T16:12:54.013 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-04-02T16:12:54.015 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-04-02T16:12:54.030 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.0dpEhCq9It 2024-04-02T16:12:54.030 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0dpEhCq9It 2024-04-02T16:12:54.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.0dpEhCq9It 2024-04-02T16:12:54.236 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.0dpEhCq9It is 6KB 2024-04-02T16:12:54.284 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.0dpEhCq9It 2024-04-02T16:12:54.297 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T16:12:54.362 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T16:12:54.362 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T16:12:54.362 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T16:12:54.362 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T16:12:54.363 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T16:12:54.501 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-04-02T16:13:06.914 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-02T16:13:06.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T16:13:06.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:13:24.510 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-02T16:13:24.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T16:13:24.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:13:25.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-04-02T16:13:34.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T16:13:34.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:13:37.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-04-02T16:13:43.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-04-02T16:13:49.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T16:13:49.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:13:52.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T16:13:52.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:13:55.138 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-04-02T16:14:07.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T16:14:07.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:14:21.338 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-04-02T16:14:21.686 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T16:14:21.686 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-434.el9.x86_64"... 2024-04-02T16:14:21.686 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-04-02T16:14:21.702 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-434.el9.x86_64 2024-04-02T16:14:21.702 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T16:14:21.702 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T16:14:21.702 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T16:14:22.703 DEBUG:teuthology.parallel:result is None 2024-04-02T16:14:38.998 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-04-02T16:14:39.429 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T16:14:39.429 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-434.el9.x86_64"... 2024-04-02T16:14:39.429 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-04-02T16:14:39.443 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-434.el9.x86_64 2024-04-02T16:14:39.443 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T16:14:39.443 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T16:14:39.443 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T16:14:40.444 DEBUG:teuthology.parallel:result is None 2024-04-02T16:14:40.445 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T16:14:40.452 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T16:14:40.452 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T16:14:40.455 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T16:14:40.481 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T16:14:40.489 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T16:14:40.548 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T16:14:40.548 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T16:14:40.550 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T16:14:40.592 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T16:14:40.602 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T16:14:40.603 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T16:14:40.615 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T16:14:40.704 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:14:40.714 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:14:40.744 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:14:40.778 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:14:40.781 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T16:14:40.788 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T16:14:40.788 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T16:14:40.791 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T16:14:40.839 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T16:14:40.848 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T16:14:40.848 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T16:14:40.862 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T16:14:40.886 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:14:40.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:14:41.013 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:14:41.094 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:14:41.140 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:14:41.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T16:14:41.208 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:14:41.247 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:14:41.302 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:14:41.380 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:14:41.425 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:14:41.425 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T16:14:41.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-04-02T16:14:41.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-04-02T16:14:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T16:14:41.721 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T16:14:42.253 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T16:14:42.303 INFO:teuthology.task.internal:Starting timer... 2024-04-02T16:14:42.304 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T16:14:42.410 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T16:14:42.457 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T16:14:42.457 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-04-02T16:14:42.524 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-04-02T16:14:42.526 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-04-02T16:14:42.588 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-04-02T16:14:42.590 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T16:14:42.590 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-04-02T16:14:42.612 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-04-02T16:14:42.612 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-04-02T16:14:42.636 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-04-02T16:14:42.636 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-04-02T16:14:42.636 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T16:14:42.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:14:42.681 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T16:14:42.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:14:42.710 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T16:14:42.711 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-04-02T16:14:42.764 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-04-02T16:14:42.803 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T16:14:42.849 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-02T16:14:42.854 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-02T16:14:42.993 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T16:14:43.009 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T16:14:43.011 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi050.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T16:19:10.565 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-04-02T16:19:10.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T16:19:10.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:19:10.662 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-04-02T16:19:10.737 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T16:19:10.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T16:19:10.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:19:10.826 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-04-02T16:19:10.904 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T16:19:10.905 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T16:19:10.915 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T16:19:10.916 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T16:19:10.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:19:10.919 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T16:19:10.919 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:19:10.947 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T16:19:10.964 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T16:19:10.976 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T16:19:10.992 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T16:19:10.994 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-04-02T16:19:11.008 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-04-02T16:19:11.021 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-04-02T16:19:11.025 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T16:19:11.035 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-04-02T16:19:11.041 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T16:19:11.052 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T16:19:11.067 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T16:19:11.093 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-04-02T16:19:11.096 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T16:19:11.097 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-04-02T16:19:11.097 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.097 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.097 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.097 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.119 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-04-02T16:19:11.123 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T16:19:11.123 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-04-02T16:19:11.123 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.123 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.123 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.123 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T16:19:11.124 INFO:teuthology.run_tasks:Running task install... 2024-04-02T16:19:11.132 DEBUG:teuthology.task.install:project ceph 2024-04-02T16:19:11.133 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}} 2024-04-02T16:19:11.133 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'} 2024-04-02T16:19:11.133 INFO:teuthology.task.install:Using flavor: default 2024-04-02T16:19:11.138 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T16:19:11.139 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-02T16:19:11.139 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T16:19:11.140 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:19:11.141 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T16:19:11.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:19:11.358 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T16:19:11.358 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T16:19:11.373 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T16:19:11.373 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T16:19:11.478 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T16:19:11.478 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:19:11.478 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T16:19:11.508 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T16:19:11.508 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T16:19:11.544 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T16:19:11.544 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:19:11.544 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T16:19:11.573 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T16:19:11.573 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T16:19:11.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T16:19:11.640 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T16:19:11.657 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T16:19:11.721 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T16:19:11.725 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-04-02T16:19:11.727 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-04-02T16:19:11.787 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-04-02T16:19:11.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-04-02T16:19:11.992 INFO:teuthology.orchestra.run.smithi050.stdout:74 files removed 2024-04-02T16:19:12.019 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T16:19:12.043 INFO:teuthology.orchestra.run.smithi188.stdout:74 files removed 2024-04-02T16:19:12.070 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T16:19:12.780 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-02T16:19:12.797 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-04-02T16:19:13.148 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-02T16:19:13.190 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-02T16:19:13.648 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-02T16:19:13.710 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-02T16:19:14.034 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-04-02T16:19:14.159 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-04-02T16:19:15.475 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-04-02T16:19:15.626 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-04-02T16:19:16.350 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-02T16:19:16.375 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.0 MB 00:00 2024-04-02T16:19:18.176 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-02T16:19:20.919 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-04-02T16:19:22.193 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-04-02T16:19:23.610 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-02T16:19:24.956 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-02T16:19:26.986 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-04-02T16:19:28.595 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-02T16:19:30.019 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-02T16:19:30.257 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-02T16:19:32.592 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:19:32.592 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:19:32.605 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T16:19:32.607 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T16:19:32.607 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T16:19:32.608 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T16:19:32.610 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T16:19:32.614 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T16:19:32.615 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T16:19:32.695 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-02T16:19:32.707 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-04-02T16:19:32.708 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-04-02T16:19:32.708 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-04-02T16:19:32.708 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-04-02T16:19:32.708 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T16:19:32.708 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T16:19:32.708 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T16:19:32.709 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T16:19:32.709 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T16:19:32.709 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T16:19:32.709 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T16:19:32.709 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T16:19:32.709 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T16:19:32.710 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T16:19:32.710 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T16:19:32.710 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T16:19:32.710 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T16:19:32.710 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T16:19:32.711 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T16:19:32.711 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T16:19:32.711 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T16:19:32.711 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T16:19:32.711 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T16:19:32.711 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T16:19:32.712 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T16:19:32.712 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T16:19:32.712 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T16:19:32.712 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T16:19:32.712 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T16:19:32.713 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T16:19:32.713 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T16:19:32.713 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T16:19:32.713 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T16:19:32.713 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T16:19:32.713 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T16:19:32.714 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T16:19:32.714 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T16:19:32.714 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T16:19:32.714 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:19:32.714 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T16:19:32.714 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T16:19:32.715 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-04-02T16:19:32.715 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T16:19:32.715 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T16:19:32.715 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T16:19:32.715 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T16:19:32.715 INFO:teuthology.orchestra.run.smithi188.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T16:19:32.715 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T16:19:32.716 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T16:19:32.716 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T16:19:32.716 INFO:teuthology.orchestra.run.smithi188.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T16:19:32.716 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T16:19:32.716 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T16:19:32.716 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T16:19:32.716 INFO:teuthology.orchestra.run.smithi188.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T16:19:32.717 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T16:19:32.717 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:19:32.717 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T16:19:32.717 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T16:19:32.717 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T16:19:32.717 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T16:19:32.717 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T16:19:32.718 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T16:19:32.718 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T16:19:32.718 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T16:19:32.718 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T16:19:32.718 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-04-02T16:19:32.718 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T16:19:32.718 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T16:19:32.719 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T16:19:32.719 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T16:19:32.719 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T16:19:32.719 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T16:19:32.719 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T16:19:32.719 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T16:19:32.719 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T16:19:32.720 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T16:19:32.720 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T16:19:32.720 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T16:19:32.720 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T16:19:32.720 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T16:19:32.720 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T16:19:32.720 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T16:19:32.721 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T16:19:32.721 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T16:19:32.721 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T16:19:32.721 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T16:19:32.721 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T16:19:32.721 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T16:19:32.721 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T16:19:32.722 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T16:19:32.722 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T16:19:32.722 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T16:19:32.722 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T16:19:32.722 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T16:19:32.722 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T16:19:32.722 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T16:19:32.723 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T16:19:32.724 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T16:19:32.725 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T16:19:32.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T16:19:32.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T16:19:32.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T16:19:32.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T16:19:32.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T16:19:32.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T16:19:32.726 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T16:19:32.727 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T16:19:32.728 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T16:19:32.729 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-04-02T16:19:32.730 INFO:teuthology.orchestra.run.smithi188.stdout:Install 150 Packages 2024-04-02T16:19:32.731 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 24 Packages 2024-04-02T16:19:32.731 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:19:32.731 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 226 M 2024-04-02T16:19:32.731 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-04-02T16:19:33.522 INFO:teuthology.orchestra.run.smithi188.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-02T16:19:34.020 INFO:teuthology.orchestra.run.smithi188.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.7 MB/s | 890 kB 00:00 2024-04-02T16:19:34.095 INFO:teuthology.orchestra.run.smithi188.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 1.9 MB/s | 145 kB 00:00 2024-04-02T16:19:34.904 INFO:teuthology.orchestra.run.smithi188.stdout:(4/174): ceph-common-19.0.0-2609.g191d963a.el9. 13 MB/s | 21 MB 00:01 2024-04-02T16:19:34.979 INFO:teuthology.orchestra.run.smithi188.stdout:(5/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-04-02T16:19:35.029 INFO:teuthology.orchestra.run.smithi188.stdout:(6/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-04-02T16:19:35.071 INFO:teuthology.orchestra.run.smithi188.stdout:(7/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-04-02T16:19:35.372 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-02T16:19:35.663 INFO:teuthology.orchestra.run.smithi188.stdout:(8/174): ceph-radosgw-19.0.0-2609.g191d963a.el9 17 MB/s | 9.8 MB 00:00 2024-04-02T16:19:35.705 INFO:teuthology.orchestra.run.smithi188.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 608 kB/s | 25 kB 00:00 2024-04-02T16:19:35.956 INFO:teuthology.orchestra.run.smithi188.stdout:(10/174): ceph-mon-19.0.0-2609.g191d963a.el9.x8 4.8 MB/s | 4.7 MB 00:00 2024-04-02T16:19:36.081 INFO:teuthology.orchestra.run.smithi188.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 1.8 MB/s | 225 kB 00:00 2024-04-02T16:19:36.198 INFO:teuthology.orchestra.run.smithi188.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 232 kB/s | 27 kB 00:00 2024-04-02T16:19:36.298 INFO:teuthology.orchestra.run.smithi188.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 318 kB/s | 32 kB 00:00 2024-04-02T16:19:36.498 INFO:teuthology.orchestra.run.smithi188.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 3.7 MB/s | 747 kB 00:00 2024-04-02T16:19:36.607 INFO:teuthology.orchestra.run.smithi188.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.5 MB/s | 164 kB 00:00 2024-04-02T16:19:36.723 INFO:teuthology.orchestra.run.smithi188.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.1 MB/s | 127 kB 00:00 2024-04-02T16:19:36.857 INFO:teuthology.orchestra.run.smithi188.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 3.7 MB/s | 500 kB 00:00 2024-04-02T16:19:37.649 INFO:teuthology.orchestra.run.smithi188.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 6.5 MB/s | 5.1 MB 00:00 2024-04-02T16:19:37.732 INFO:teuthology.orchestra.run.smithi188.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 543 kB/s | 45 kB 00:00 2024-04-02T16:19:37.824 INFO:teuthology.orchestra.run.smithi188.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.4 MB/s | 134 kB 00:00 2024-04-02T16:19:38.000 INFO:teuthology.orchestra.run.smithi188.stdout:(21/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 5.7 MB/s | 17 MB 00:02 2024-04-02T16:19:38.025 INFO:teuthology.orchestra.run.smithi188.stdout:(22/174): python3-cephfs-19.0.0-2609.g191d963a. 806 kB/s | 161 kB 00:00 2024-04-02T16:19:38.050 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:19:38.050 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T16:19:38.066 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T16:19:38.067 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T16:19:38.068 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T16:19:38.070 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T16:19:38.072 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T16:19:38.075 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T16:19:38.076 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T16:19:38.092 INFO:teuthology.orchestra.run.smithi188.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 4.4 MB/s | 299 kB 00:00 2024-04-02T16:19:38.125 INFO:teuthology.orchestra.run.smithi188.stdout:(24/174): python3-rados-19.0.0-2609.g191d963a.e 2.5 MB/s | 320 kB 00:00 2024-04-02T16:19:38.161 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-04-02T16:19:38.167 INFO:teuthology.orchestra.run.smithi188.stdout:(25/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.3 MB/s | 99 kB 00:00 2024-04-02T16:19:38.179 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-04-02T16:19:38.179 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-04-02T16:19:38.179 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T16:19:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T16:19:38.181 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T16:19:38.182 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T16:19:38.183 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T16:19:38.184 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T16:19:38.185 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T16:19:38.186 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T16:19:38.187 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T16:19:38.188 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T16:19:38.189 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T16:19:38.190 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T16:19:38.191 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-04-02T16:19:38.192 INFO:teuthology.orchestra.run.smithi050.stdout:Install 150 Packages 2024-04-02T16:19:38.193 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 24 Packages 2024-04-02T16:19:38.193 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:19:38.193 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 226 M 2024-04-02T16:19:38.193 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-04-02T16:19:38.201 INFO:teuthology.orchestra.run.smithi188.stdout:(26/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-02T16:19:38.676 INFO:teuthology.orchestra.run.smithi188.stdout:(27/174): ceph-test-19.0.0-2609.g191d963a.el9.x 16 MB/s | 48 MB 00:02 2024-04-02T16:19:38.701 INFO:teuthology.orchestra.run.smithi188.stdout:(28/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 340 kB/s | 170 kB 00:00 2024-04-02T16:19:38.727 INFO:teuthology.orchestra.run.smithi188.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2609.g 681 kB/s | 34 kB 00:00 2024-04-02T16:19:38.785 INFO:teuthology.orchestra.run.smithi188.stdout:(30/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 5.0 MB/s | 3.1 MB 00:00 2024-04-02T16:19:38.807 INFO:teuthology.orchestra.run.smithi050.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-02T16:19:38.810 INFO:teuthology.orchestra.run.smithi188.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.3 MB/s | 141 kB 00:00 2024-04-02T16:19:38.911 INFO:teuthology.orchestra.run.smithi188.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 2.4 MB/s | 249 kB 00:00 2024-04-02T16:19:39.078 INFO:teuthology.orchestra.run.smithi188.stdout:(33/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 10 MB/s | 3.5 MB 00:00 2024-04-02T16:19:39.103 INFO:teuthology.orchestra.run.smithi188.stdout:(34/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 257 kB/s | 49 kB 00:00 2024-04-02T16:19:39.137 INFO:teuthology.orchestra.run.smithi188.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2609.g1 281 kB/s | 16 kB 00:00 2024-04-02T16:19:39.187 INFO:teuthology.orchestra.run.smithi188.stdout:(36/174): ceph-volume-19.0.0-2609.g191d963a.el9 3.1 MB/s | 264 kB 00:00 2024-04-02T16:19:39.221 INFO:teuthology.orchestra.run.smithi188.stdout:(37/174): cephadm-19.0.0-2609.g191d963a.el9.noa 8.9 MB/s | 762 kB 00:00 2024-04-02T16:19:39.279 INFO:teuthology.orchestra.run.smithi188.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 199 kB/s | 18 kB 00:00 2024-04-02T16:19:39.379 INFO:teuthology.orchestra.run.smithi188.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 4.1 MB/s | 662 kB 00:00 2024-04-02T16:19:39.413 INFO:teuthology.orchestra.run.smithi188.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.8 MB/s | 920 kB 00:00 2024-04-02T16:19:39.423 INFO:teuthology.orchestra.run.smithi050.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.4 MB/s | 890 kB 00:00 2024-04-02T16:19:39.438 INFO:teuthology.orchestra.run.smithi188.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-04-02T16:19:39.464 INFO:teuthology.orchestra.run.smithi188.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-02T16:19:39.489 INFO:teuthology.orchestra.run.smithi188.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-04-02T16:19:39.524 INFO:teuthology.orchestra.run.smithi050.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 1.4 MB/s | 145 kB 00:00 2024-04-02T16:19:39.589 INFO:teuthology.orchestra.run.smithi188.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2024-04-02T16:19:39.639 INFO:teuthology.orchestra.run.smithi188.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-02T16:19:39.664 INFO:teuthology.orchestra.run.smithi188.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 99 kB/s | 17 kB 00:00 2024-04-02T16:19:39.690 INFO:teuthology.orchestra.run.smithi188.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 10 kB 00:00 2024-04-02T16:19:39.715 INFO:teuthology.orchestra.run.smithi188.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-04-02T16:19:39.741 INFO:teuthology.orchestra.run.smithi188.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-04-02T16:19:39.774 INFO:teuthology.orchestra.run.smithi188.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-04-02T16:19:39.799 INFO:teuthology.orchestra.run.smithi188.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 578 kB 00:00 2024-04-02T16:19:39.824 INFO:teuthology.orchestra.run.smithi188.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 46 kB 00:00 2024-04-02T16:19:39.866 INFO:teuthology.orchestra.run.smithi188.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-04-02T16:19:39.891 INFO:teuthology.orchestra.run.smithi188.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-04-02T16:19:39.908 INFO:teuthology.orchestra.run.smithi188.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 829 kB/s | 34 kB 00:00 2024-04-02T16:19:39.933 INFO:teuthology.orchestra.run.smithi188.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-04-02T16:19:40.017 INFO:teuthology.orchestra.run.smithi188.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-02T16:19:40.059 INFO:teuthology.orchestra.run.smithi188.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-04-02T16:19:40.084 INFO:teuthology.orchestra.run.smithi188.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 817 kB 00:00 2024-04-02T16:19:40.109 INFO:teuthology.orchestra.run.smithi188.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 132 kB 00:00 2024-04-02T16:19:40.141 INFO:teuthology.orchestra.run.smithi050.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 3.8 MB/s | 2.4 MB 00:00 2024-04-02T16:19:40.151 INFO:teuthology.orchestra.run.smithi188.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-02T16:19:40.260 INFO:teuthology.orchestra.run.smithi188.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-04-02T16:19:40.277 INFO:teuthology.orchestra.run.smithi188.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 465 kB/s | 78 kB 00:00 2024-04-02T16:19:40.360 INFO:teuthology.orchestra.run.smithi188.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-04-02T16:19:40.386 INFO:teuthology.orchestra.run.smithi188.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 199 kB/s | 24 kB 00:00 2024-04-02T16:19:40.411 INFO:teuthology.orchestra.run.smithi188.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-04-02T16:19:40.433 INFO:teuthology.orchestra.run.smithi050.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 5.1 MB/s | 1.5 MB 00:00 2024-04-02T16:19:40.436 INFO:teuthology.orchestra.run.smithi188.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-02T16:19:40.461 INFO:teuthology.orchestra.run.smithi188.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-04-02T16:19:40.568 INFO:teuthology.orchestra.run.smithi188.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 4.8 MB 00:00 2024-04-02T16:19:40.595 INFO:teuthology.orchestra.run.smithi188.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-04-02T16:19:40.620 INFO:teuthology.orchestra.run.smithi188.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-04-02T16:19:40.646 INFO:teuthology.orchestra.run.smithi188.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 897 kB/s | 69 kB 00:00 2024-04-02T16:19:40.658 INFO:teuthology.orchestra.run.smithi050.stdout:(6/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-02T16:19:40.671 INFO:teuthology.orchestra.run.smithi188.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-04-02T16:19:40.696 INFO:teuthology.orchestra.run.smithi188.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-02T16:19:40.721 INFO:teuthology.orchestra.run.smithi188.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-02T16:19:40.747 INFO:teuthology.orchestra.run.smithi188.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-04-02T16:19:40.772 INFO:teuthology.orchestra.run.smithi188.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-02T16:19:40.797 INFO:teuthology.orchestra.run.smithi188.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 568 kB/s | 43 kB 00:00 2024-04-02T16:19:40.964 INFO:teuthology.orchestra.run.smithi188.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 237 kB/s | 39 kB 00:00 2024-04-02T16:19:40.989 INFO:teuthology.orchestra.run.smithi188.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 218 kB/s | 41 kB 00:00 2024-04-02T16:19:41.148 INFO:teuthology.orchestra.run.smithi188.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-04-02T16:19:41.173 INFO:teuthology.orchestra.run.smithi188.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2024-04-02T16:19:41.198 INFO:teuthology.orchestra.run.smithi188.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 168 kB/s | 35 kB 00:00 2024-04-02T16:19:41.223 INFO:teuthology.orchestra.run.smithi188.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 929 kB/s | 46 kB 00:00 2024-04-02T16:19:41.257 INFO:teuthology.orchestra.run.smithi188.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-04-02T16:19:41.282 INFO:teuthology.orchestra.run.smithi188.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 675 kB/s | 56 kB 00:00 2024-04-02T16:19:41.308 INFO:teuthology.orchestra.run.smithi188.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-02T16:19:41.317 INFO:teuthology.orchestra.run.smithi050.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-04-02T16:19:41.333 INFO:teuthology.orchestra.run.smithi188.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-04-02T16:19:41.358 INFO:teuthology.orchestra.run.smithi188.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-02T16:19:41.383 INFO:teuthology.orchestra.run.smithi188.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T16:19:41.417 INFO:teuthology.orchestra.run.smithi188.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-02T16:19:41.442 INFO:teuthology.orchestra.run.smithi188.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-04-02T16:19:41.467 INFO:teuthology.orchestra.run.smithi188.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-02T16:19:41.501 INFO:teuthology.orchestra.run.smithi188.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-04-02T16:19:42.309 INFO:teuthology.orchestra.run.smithi188.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 216 kB/s | 187 kB 00:00 2024-04-02T16:19:42.359 INFO:teuthology.orchestra.run.smithi188.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 227 kB/s | 195 kB 00:00 2024-04-02T16:19:42.451 INFO:teuthology.orchestra.run.smithi188.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 235 kB/s | 21 kB 00:00 2024-04-02T16:19:42.502 INFO:teuthology.orchestra.run.smithi188.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-02T16:19:43.077 INFO:teuthology.orchestra.run.smithi188.stdout:(99/174): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-04-02T16:19:43.110 INFO:teuthology.orchestra.run.smithi188.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 489 kB/s | 803 kB 00:01 2024-04-02T16:19:43.259 INFO:teuthology.orchestra.run.smithi050.stdout:(8/174): ceph-common-19.0.0-2609.g191d963a.el9. 4.4 MB/s | 21 MB 00:04 2024-04-02T16:19:43.286 INFO:teuthology.orchestra.run.smithi188.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.2 MB/s | 245 kB 00:00 2024-04-02T16:19:43.311 INFO:teuthology.orchestra.run.smithi188.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.2 MB/s | 248 kB 00:00 2024-04-02T16:19:43.336 INFO:teuthology.orchestra.run.smithi188.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-02T16:19:43.367 INFO:teuthology.orchestra.run.smithi050.stdout:(9/174): ceph-radosgw-19.0.0-2609.g191d963a.el9 4.8 MB/s | 9.8 MB 00:02 2024-04-02T16:19:43.392 INFO:teuthology.orchestra.run.smithi050.stdout:(10/174): ceph-selinux-19.0.0-2609.g191d963a.el 188 kB/s | 25 kB 00:00 2024-04-02T16:19:43.568 INFO:teuthology.orchestra.run.smithi050.stdout:(11/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 5.8 MB/s | 17 MB 00:02 2024-04-02T16:19:43.610 INFO:teuthology.orchestra.run.smithi188.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-04-02T16:19:43.610 INFO:teuthology.orchestra.run.smithi050.stdout:(12/174): cephfs-mirror-19.0.0-2609.g191d963a.e 1.1 MB/s | 225 kB 00:00 2024-04-02T16:19:43.629 INFO:teuthology.orchestra.run.smithi188.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.7 MB/s | 4.1 MB 00:01 2024-04-02T16:19:43.661 INFO:teuthology.orchestra.run.smithi050.stdout:(13/174): cephfs-shell-19.0.0-2609.g191d963a.el 295 kB/s | 27 kB 00:00 2024-04-02T16:19:43.679 INFO:teuthology.orchestra.run.smithi050.stdout:(14/174): libcephfs-devel-19.0.0-2609.g191d963a 379 kB/s | 32 kB 00:00 2024-04-02T16:19:43.786 INFO:teuthology.orchestra.run.smithi050.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.5 MB/s | 164 kB 00:00 2024-04-02T16:19:43.845 INFO:teuthology.orchestra.run.smithi050.stdout:(16/174): libcephfs2-19.0.0-2609.g191d963a.el9. 4.0 MB/s | 747 kB 00:00 2024-04-02T16:19:43.854 INFO:teuthology.orchestra.run.smithi188.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-02T16:19:43.879 INFO:teuthology.orchestra.run.smithi188.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-02T16:19:43.886 INFO:teuthology.orchestra.run.smithi050.stdout:(17/174): librados-devel-19.0.0-2609.g191d963a. 1.2 MB/s | 127 kB 00:00 2024-04-02T16:19:43.904 INFO:teuthology.orchestra.run.smithi188.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-04-02T16:19:43.970 INFO:teuthology.orchestra.run.smithi050.stdout:(18/174): libradosstriper1-19.0.0-2609.g191d963 3.9 MB/s | 500 kB 00:00 2024-04-02T16:19:44.046 INFO:teuthology.orchestra.run.smithi188.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-02T16:19:44.071 INFO:teuthology.orchestra.run.smithi050.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 452 kB/s | 45 kB 00:00 2024-04-02T16:19:44.096 INFO:teuthology.orchestra.run.smithi188.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 136 kB/s | 26 kB 00:00 2024-04-02T16:19:44.122 INFO:teuthology.orchestra.run.smithi188.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-04-02T16:19:44.147 INFO:teuthology.orchestra.run.smithi188.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-02T16:19:44.172 INFO:teuthology.orchestra.run.smithi188.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-02T16:19:44.178 INFO:teuthology.orchestra.run.smithi050.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.2 MB/s | 134 kB 00:00 2024-04-02T16:19:44.206 INFO:teuthology.orchestra.run.smithi188.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-04-02T16:19:44.231 INFO:teuthology.orchestra.run.smithi188.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 9.3 MB/s | 548 kB 00:00 2024-04-02T16:19:44.256 INFO:teuthology.orchestra.run.smithi188.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 881 kB/s | 43 kB 00:00 2024-04-02T16:19:44.279 INFO:teuthology.orchestra.run.smithi050.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.6 MB/s | 161 kB 00:00 2024-04-02T16:19:44.281 INFO:teuthology.orchestra.run.smithi188.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-02T16:19:44.307 INFO:teuthology.orchestra.run.smithi188.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-02T16:19:44.324 INFO:teuthology.orchestra.run.smithi188.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-04-02T16:19:44.357 INFO:teuthology.orchestra.run.smithi188.stdout:(120/174): protobuf-compiler-3.14.0-13.el9.x86_ 846 kB/s | 863 kB 00:01 2024-04-02T16:19:44.383 INFO:teuthology.orchestra.run.smithi188.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-02T16:19:44.408 INFO:teuthology.orchestra.run.smithi188.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-02T16:19:44.435 INFO:teuthology.orchestra.run.smithi188.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 477 kB/s | 35 kB 00:00 2024-04-02T16:19:44.458 INFO:teuthology.orchestra.run.smithi188.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-04-02T16:19:44.500 INFO:teuthology.orchestra.run.smithi188.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-02T16:19:44.525 INFO:teuthology.orchestra.run.smithi188.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-04-02T16:19:44.551 INFO:teuthology.orchestra.run.smithi188.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 120 kB/s | 11 kB 00:00 2024-04-02T16:19:44.576 INFO:teuthology.orchestra.run.smithi188.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-04-02T16:19:44.601 INFO:teuthology.orchestra.run.smithi188.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-02T16:19:44.626 INFO:teuthology.orchestra.run.smithi188.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-02T16:19:44.652 INFO:teuthology.orchestra.run.smithi188.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-02T16:19:44.685 INFO:teuthology.orchestra.run.smithi188.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-02T16:19:44.710 INFO:teuthology.orchestra.run.smithi188.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 562 kB/s | 46 kB 00:00 2024-04-02T16:19:44.736 INFO:teuthology.orchestra.run.smithi188.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 953 kB/s | 79 kB 00:00 2024-04-02T16:19:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 988 kB/s | 58 kB 00:00 2024-04-02T16:19:44.795 INFO:teuthology.orchestra.run.smithi188.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-02T16:19:44.820 INFO:teuthology.orchestra.run.smithi188.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-02T16:19:44.845 INFO:teuthology.orchestra.run.smithi188.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-02T16:19:44.871 INFO:teuthology.orchestra.run.smithi188.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-02T16:19:44.896 INFO:teuthology.orchestra.run.smithi188.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-02T16:19:44.904 INFO:teuthology.orchestra.run.smithi050.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 512 kB/s | 320 kB 00:00 2024-04-02T16:19:44.921 INFO:teuthology.orchestra.run.smithi188.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-02T16:19:44.946 INFO:teuthology.orchestra.run.smithi188.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 788 kB/s | 59 kB 00:00 2024-04-02T16:19:44.971 INFO:teuthology.orchestra.run.smithi188.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-02T16:19:44.997 INFO:teuthology.orchestra.run.smithi188.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-02T16:19:45.021 INFO:teuthology.orchestra.run.smithi050.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 2.5 MB/s | 299 kB 00:00 2024-04-02T16:19:45.022 INFO:teuthology.orchestra.run.smithi188.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-02T16:19:45.047 INFO:teuthology.orchestra.run.smithi188.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-02T16:19:45.076 INFO:teuthology.orchestra.run.smithi188.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-04-02T16:19:45.089 INFO:teuthology.orchestra.run.smithi188.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-02T16:19:45.114 INFO:teuthology.orchestra.run.smithi188.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-02T16:19:45.129 INFO:teuthology.orchestra.run.smithi050.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 919 kB/s | 99 kB 00:00 2024-04-02T16:19:45.164 INFO:teuthology.orchestra.run.smithi188.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-02T16:19:45.238 INFO:teuthology.orchestra.run.smithi050.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 790 kB/s | 85 kB 00:00 2024-04-02T16:19:45.248 INFO:teuthology.orchestra.run.smithi188.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 27 MB/s | 2.3 MB 00:00 2024-04-02T16:19:45.323 INFO:teuthology.orchestra.run.smithi188.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-02T16:19:45.357 INFO:teuthology.orchestra.run.smithi188.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-02T16:19:45.407 INFO:teuthology.orchestra.run.smithi188.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-04-02T16:19:45.449 INFO:teuthology.orchestra.run.smithi188.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-02T16:19:45.664 INFO:teuthology.orchestra.run.smithi050.stdout:(26/174): librgw2-19.0.0-2609.g191d963a.el9.x86 2.9 MB/s | 5.1 MB 00:01 2024-04-02T16:19:45.780 INFO:teuthology.orchestra.run.smithi050.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.4 MB/s | 170 kB 00:00 2024-04-02T16:19:45.914 INFO:teuthology.orchestra.run.smithi050.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 254 kB/s | 34 kB 00:00 2024-04-02T16:19:45.950 INFO:teuthology.orchestra.run.smithi188.stdout:(156/174): librbd1-19.0.0-2609.g191d963a.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-02T16:19:45.998 INFO:teuthology.orchestra.run.smithi050.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.7 MB/s | 141 kB 00:00 2024-04-02T16:19:46.133 INFO:teuthology.orchestra.run.smithi188.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 104 kB/s | 71 kB 00:00 2024-04-02T16:19:46.198 INFO:teuthology.orchestra.run.smithi050.stdout:(30/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 3.2 MB/s | 3.1 MB 00:00 2024-04-02T16:19:46.550 INFO:teuthology.orchestra.run.smithi188.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 61 kB/s | 36 kB 00:00 2024-04-02T16:19:46.983 INFO:teuthology.orchestra.run.smithi188.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 226 kB/s | 192 kB 00:00 2024-04-02T16:19:47.023 INFO:teuthology.orchestra.run.smithi050.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 3.5 MB/s | 3.5 MB 00:01 2024-04-02T16:19:47.115 INFO:teuthology.orchestra.run.smithi050.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 2.7 MB/s | 249 kB 00:00 2024-04-02T16:19:47.199 INFO:teuthology.orchestra.run.smithi050.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 592 kB/s | 49 kB 00:00 2024-04-02T16:19:47.283 INFO:teuthology.orchestra.run.smithi050.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 197 kB/s | 16 kB 00:00 2024-04-02T16:19:47.400 INFO:teuthology.orchestra.run.smithi050.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 2.2 MB/s | 264 kB 00:00 2024-04-02T16:19:47.484 INFO:teuthology.orchestra.run.smithi188.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 274 kB/s | 256 kB 00:00 2024-04-02T16:19:47.575 INFO:teuthology.orchestra.run.smithi188.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 39 kB/s | 23 kB 00:00 2024-04-02T16:19:47.691 INFO:teuthology.orchestra.run.smithi050.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 2.6 MB/s | 762 kB 00:00 2024-04-02T16:19:47.892 INFO:teuthology.orchestra.run.smithi050.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 91 kB/s | 18 kB 00:00 2024-04-02T16:19:48.026 INFO:teuthology.orchestra.run.smithi050.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-04-02T16:19:48.076 INFO:teuthology.orchestra.run.smithi050.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-04-02T16:19:48.110 INFO:teuthology.orchestra.run.smithi050.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-02T16:19:48.160 INFO:teuthology.orchestra.run.smithi050.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-02T16:19:48.185 INFO:teuthology.orchestra.run.smithi050.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-04-02T16:19:48.251 INFO:teuthology.orchestra.run.smithi188.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 141 kB/s | 108 kB 00:00 2024-04-02T16:19:48.278 INFO:teuthology.orchestra.run.smithi050.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-02T16:19:48.302 INFO:teuthology.orchestra.run.smithi050.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 690 kB/s | 17 kB 00:00 2024-04-02T16:19:48.328 INFO:teuthology.orchestra.run.smithi050.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 400 kB/s | 10 kB 00:00 2024-04-02T16:19:48.361 INFO:teuthology.orchestra.run.smithi050.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-04-02T16:19:48.395 INFO:teuthology.orchestra.run.smithi050.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-02T16:19:48.437 INFO:teuthology.orchestra.run.smithi050.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 661 kB 00:00 2024-04-02T16:19:48.460 INFO:teuthology.orchestra.run.smithi188.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 175 kB/s | 155 kB 00:00 2024-04-02T16:19:48.479 INFO:teuthology.orchestra.run.smithi050.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-04-02T16:19:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-02T16:19:48.555 INFO:teuthology.orchestra.run.smithi050.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-04-02T16:19:48.596 INFO:teuthology.orchestra.run.smithi050.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-04-02T16:19:48.639 INFO:teuthology.orchestra.run.smithi050.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-04-02T16:19:48.672 INFO:teuthology.orchestra.run.smithi050.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-04-02T16:19:48.760 INFO:teuthology.orchestra.run.smithi188.stdout:(164/174): librados2-19.0.0-2609.g191d963a.el9. 947 kB/s | 3.4 MB 00:03 2024-04-02T16:19:48.798 INFO:teuthology.orchestra.run.smithi050.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-02T16:19:48.890 INFO:teuthology.orchestra.run.smithi050.stdout:(56/174): ceph-mgr-diskprediction-local-19.0.0- 2.8 MB/s | 7.4 MB 00:02 2024-04-02T16:19:48.931 INFO:teuthology.orchestra.run.smithi050.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-04-02T16:19:48.974 INFO:teuthology.orchestra.run.smithi050.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 9.8 MB/s | 817 kB 00:00 2024-04-02T16:19:48.999 INFO:teuthology.orchestra.run.smithi050.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 132 kB 00:00 2024-04-02T16:19:49.118 INFO:teuthology.orchestra.run.smithi188.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 156 kB/s | 136 kB 00:00 2024-04-02T16:19:49.133 INFO:teuthology.orchestra.run.smithi050.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-04-02T16:19:49.152 INFO:teuthology.orchestra.run.smithi188.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 92 kB/s | 64 kB 00:00 2024-04-02T16:19:49.169 INFO:teuthology.orchestra.run.smithi050.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-02T16:19:49.192 INFO:teuthology.orchestra.run.smithi050.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-02T16:19:49.368 INFO:teuthology.orchestra.run.smithi050.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 138 kB/s | 24 kB 00:00 2024-04-02T16:19:49.459 INFO:teuthology.orchestra.run.smithi050.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-04-02T16:19:49.484 INFO:teuthology.orchestra.run.smithi050.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 158 kB/s | 18 kB 00:00 2024-04-02T16:19:49.485 INFO:teuthology.orchestra.run.smithi188.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 40 kB/s | 29 kB 00:00 2024-04-02T16:19:49.510 INFO:teuthology.orchestra.run.smithi050.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-02T16:19:49.535 INFO:teuthology.orchestra.run.smithi050.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-04-02T16:19:49.710 INFO:teuthology.orchestra.run.smithi050.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-04-02T16:19:49.735 INFO:teuthology.orchestra.run.smithi050.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-04-02T16:19:49.762 INFO:teuthology.orchestra.run.smithi050.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-02T16:19:49.786 INFO:teuthology.orchestra.run.smithi050.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-02T16:19:49.811 INFO:teuthology.orchestra.run.smithi050.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-04-02T16:19:49.836 INFO:teuthology.orchestra.run.smithi050.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-02T16:19:49.853 INFO:teuthology.orchestra.run.smithi050.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-04-02T16:19:49.879 INFO:teuthology.orchestra.run.smithi050.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-04-02T16:19:49.904 INFO:teuthology.orchestra.run.smithi050.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 948 kB/s | 47 kB 00:00 2024-04-02T16:19:49.930 INFO:teuthology.orchestra.run.smithi050.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 851 kB/s | 43 kB 00:00 2024-04-02T16:19:50.130 INFO:teuthology.orchestra.run.smithi050.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 194 kB/s | 39 kB 00:00 2024-04-02T16:19:50.389 INFO:teuthology.orchestra.run.smithi050.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-04-02T16:19:50.414 INFO:teuthology.orchestra.run.smithi050.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 145 kB/s | 41 kB 00:00 2024-04-02T16:19:50.447 INFO:teuthology.orchestra.run.smithi050.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-02T16:19:50.472 INFO:teuthology.orchestra.run.smithi050.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 595 kB/s | 35 kB 00:00 2024-04-02T16:19:50.498 INFO:teuthology.orchestra.run.smithi050.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 924 kB/s | 46 kB 00:00 2024-04-02T16:19:50.532 INFO:teuthology.orchestra.run.smithi050.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-02T16:19:50.557 INFO:teuthology.orchestra.run.smithi050.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 965 kB/s | 56 kB 00:00 2024-04-02T16:19:50.582 INFO:teuthology.orchestra.run.smithi050.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-02T16:19:50.608 INFO:teuthology.orchestra.run.smithi050.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-04-02T16:19:50.625 INFO:teuthology.orchestra.run.smithi050.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-04-02T16:19:50.650 INFO:teuthology.orchestra.run.smithi050.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-04-02T16:19:50.684 INFO:teuthology.orchestra.run.smithi050.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-04-02T16:19:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-02T16:19:50.734 INFO:teuthology.orchestra.run.smithi050.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-02T16:19:50.751 INFO:teuthology.orchestra.run.smithi050.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-04-02T16:19:50.918 INFO:teuthology.orchestra.run.smithi050.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-04-02T16:19:50.985 INFO:teuthology.orchestra.run.smithi050.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-02T16:19:51.010 INFO:teuthology.orchestra.run.smithi050.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-02T16:19:51.136 INFO:teuthology.orchestra.run.smithi050.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 21 kB 00:00 2024-04-02T16:19:51.152 INFO:teuthology.orchestra.run.smithi050.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 747 kB/s | 125 kB 00:00 2024-04-02T16:19:51.261 INFO:teuthology.orchestra.run.smithi050.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 203 kB/s | 25 kB 00:00 2024-04-02T16:19:51.402 INFO:teuthology.orchestra.run.smithi188.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 543 kB/s | 1.2 MB 00:02 2024-04-02T16:19:51.561 INFO:teuthology.orchestra.run.smithi050.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 818 kB/s | 245 kB 00:00 2024-04-02T16:19:51.754 INFO:teuthology.orchestra.run.smithi050.stdout:(101/174): openssl-devel-3.0.7-27.el9.x86_64.rp 6.9 MB/s | 4.1 MB 00:00 2024-04-02T16:19:51.804 INFO:teuthology.orchestra.run.smithi050.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.0 MB/s | 248 kB 00:00 2024-04-02T16:19:51.887 INFO:teuthology.orchestra.run.smithi050.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-02T16:19:51.929 INFO:teuthology.orchestra.run.smithi050.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 535 kB/s | 94 kB 00:00 2024-04-02T16:19:52.055 INFO:teuthology.orchestra.run.smithi050.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-04-02T16:19:52.405 INFO:teuthology.orchestra.run.smithi050.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-02T16:19:52.480 INFO:teuthology.orchestra.run.smithi050.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-02T16:19:52.506 INFO:teuthology.orchestra.run.smithi050.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 685 kB/s | 309 kB 00:00 2024-04-02T16:19:52.597 INFO:teuthology.orchestra.run.smithi050.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-02T16:19:52.681 INFO:teuthology.orchestra.run.smithi050.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2024-04-02T16:19:52.744 INFO:teuthology.orchestra.run.smithi188.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 347 kB/s | 465 kB 00:01 2024-04-02T16:19:52.765 INFO:teuthology.orchestra.run.smithi050.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 809 kB/s | 67 kB 00:00 2024-04-02T16:19:52.786 INFO:teuthology.orchestra.run.smithi188.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 608 kB/s | 2.2 MB 00:03 2024-04-02T16:19:52.849 INFO:teuthology.orchestra.run.smithi050.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-02T16:19:52.907 INFO:teuthology.orchestra.run.smithi050.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-02T16:19:52.941 INFO:teuthology.orchestra.run.smithi050.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-02T16:19:53.016 INFO:teuthology.orchestra.run.smithi050.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-04-02T16:19:53.050 INFO:teuthology.orchestra.run.smithi050.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-02T16:19:53.083 INFO:teuthology.orchestra.run.smithi050.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-04-02T16:19:53.125 INFO:teuthology.orchestra.run.smithi050.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-02T16:19:53.159 INFO:teuthology.orchestra.run.smithi050.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-02T16:19:53.218 INFO:teuthology.orchestra.run.smithi050.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-02T16:19:53.251 INFO:teuthology.orchestra.run.smithi050.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-02T16:19:53.285 INFO:teuthology.orchestra.run.smithi050.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 529 kB/s | 35 kB 00:00 2024-04-02T16:19:53.327 INFO:teuthology.orchestra.run.smithi050.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-02T16:19:53.402 INFO:teuthology.orchestra.run.smithi050.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-02T16:19:53.444 INFO:teuthology.orchestra.run.smithi188.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 44 kB/s | 31 kB 00:00 2024-04-02T16:19:53.469 INFO:teuthology.orchestra.run.smithi050.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-02T16:19:53.494 INFO:teuthology.orchestra.run.smithi050.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-02T16:19:53.528 INFO:teuthology.orchestra.run.smithi188.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 897 kB/s | 74 kB 00:00 2024-04-02T16:19:53.545 INFO:teuthology.orchestra.run.smithi050.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-02T16:19:53.570 INFO:teuthology.orchestra.run.smithi050.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-02T16:19:53.620 INFO:teuthology.orchestra.run.smithi050.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-02T16:19:53.646 INFO:teuthology.orchestra.run.smithi050.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-02T16:19:53.713 INFO:teuthology.orchestra.run.smithi050.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-02T16:19:53.738 INFO:teuthology.orchestra.run.smithi050.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-04-02T16:19:53.788 INFO:teuthology.orchestra.run.smithi050.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-02T16:19:53.814 INFO:teuthology.orchestra.run.smithi050.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-04-02T16:19:53.864 INFO:teuthology.orchestra.run.smithi050.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-02T16:19:53.889 INFO:teuthology.orchestra.run.smithi050.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-02T16:19:53.939 INFO:teuthology.orchestra.run.smithi050.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-02T16:19:53.965 INFO:teuthology.orchestra.run.smithi050.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-02T16:19:54.015 INFO:teuthology.orchestra.run.smithi050.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-04-02T16:19:54.040 INFO:teuthology.orchestra.run.smithi050.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-02T16:19:54.091 INFO:teuthology.orchestra.run.smithi050.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 781 kB/s | 59 kB 00:00 2024-04-02T16:19:54.116 INFO:teuthology.orchestra.run.smithi050.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-02T16:19:54.166 INFO:teuthology.orchestra.run.smithi050.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-04-02T16:19:54.191 INFO:teuthology.orchestra.run.smithi050.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-02T16:19:54.194 INFO:teuthology.orchestra.run.smithi188.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 497 kB/s | 2.3 MB 00:04 2024-04-02T16:19:54.242 INFO:teuthology.orchestra.run.smithi050.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-02T16:19:54.284 INFO:teuthology.orchestra.run.smithi050.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-02T16:19:54.318 INFO:teuthology.orchestra.run.smithi050.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-02T16:19:54.359 INFO:teuthology.orchestra.run.smithi050.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-02T16:19:54.801 INFO:teuthology.orchestra.run.smithi050.stdout:(149/174): ceph-test-19.0.0-2609.g191d963a.el9. 4.2 MB/s | 48 MB 00:11 2024-04-02T16:19:54.835 INFO:teuthology.orchestra.run.smithi050.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-04-02T16:19:54.927 INFO:teuthology.orchestra.run.smithi050.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-02T16:19:55.002 INFO:teuthology.orchestra.run.smithi050.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-02T16:19:55.028 INFO:teuthology.orchestra.run.smithi188.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 490 kB/s | 1.1 MB 00:02 2024-04-02T16:19:55.033 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:19:55.033 INFO:teuthology.orchestra.run.smithi188.stdout:Total 10 MB/s | 226 MB 00:22 2024-04-02T16:19:55.035 INFO:teuthology.orchestra.run.smithi050.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-02T16:19:55.069 INFO:teuthology.orchestra.run.smithi050.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 7.2 MB/s | 247 kB 00:00 2024-04-02T16:19:55.111 INFO:teuthology.orchestra.run.smithi050.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-02T16:19:55.406 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-04-02T16:19:55.680 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-04-02T16:19:55.680 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-04-02T16:19:55.870 INFO:teuthology.orchestra.run.smithi050.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 94 kB/s | 71 kB 00:00 2024-04-02T16:19:55.920 INFO:teuthology.orchestra.run.smithi050.stdout:(157/174): librados2-19.0.0-2609.g191d963a.el9. 2.2 MB/s | 3.4 MB 00:01 2024-04-02T16:19:55.937 INFO:teuthology.orchestra.run.smithi050.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 547 kB/s | 36 kB 00:00 2024-04-02T16:19:56.037 INFO:teuthology.orchestra.run.smithi050.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-04-02T16:19:56.079 INFO:teuthology.orchestra.run.smithi050.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-04-02T16:19:56.096 INFO:teuthology.orchestra.run.smithi050.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 395 kB/s | 23 kB 00:00 2024-04-02T16:19:56.138 INFO:teuthology.orchestra.run.smithi050.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-02T16:19:56.205 INFO:teuthology.orchestra.run.smithi050.stdout:(163/174): librbd1-19.0.0-2609.g191d963a.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-04-02T16:19:56.230 INFO:teuthology.orchestra.run.smithi050.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-04-02T16:19:56.289 INFO:teuthology.orchestra.run.smithi050.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 901 kB/s | 136 kB 00:00 2024-04-02T16:19:56.314 INFO:teuthology.orchestra.run.smithi050.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 343 kB/s | 29 kB 00:00 2024-04-02T16:19:56.340 INFO:teuthology.orchestra.run.smithi050.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 476 kB/s | 64 kB 00:00 2024-04-02T16:19:56.398 INFO:teuthology.orchestra.run.smithi050.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-04-02T16:19:56.874 INFO:teuthology.orchestra.run.smithi050.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 979 kB/s | 465 kB 00:00 2024-04-02T16:19:57.074 INFO:teuthology.orchestra.run.smithi050.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 157 kB/s | 31 kB 00:00 2024-04-02T16:19:57.149 INFO:teuthology.orchestra.run.smithi050.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.6 MB/s | 2.2 MB 00:00 2024-04-02T16:19:57.208 INFO:teuthology.orchestra.run.smithi050.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2024-04-02T16:19:57.258 INFO:teuthology.orchestra.run.smithi050.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 2.3 MB 00:00 2024-04-02T16:19:57.283 INFO:teuthology.orchestra.run.smithi050.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 557 kB/s | 74 kB 00:00 2024-04-02T16:19:57.288 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:19:57.288 INFO:teuthology.orchestra.run.smithi050.stdout:Total 12 MB/s | 226 MB 00:19 2024-04-02T16:19:57.651 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-04-02T16:19:57.687 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-04-02T16:19:57.687 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-04-02T16:19:57.947 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-04-02T16:19:57.947 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-04-02T16:20:00.071 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-04-02T16:20:00.071 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-04-02T16:20:04.615 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-04-02T16:20:04.715 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T16:20:04.858 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T16:20:05.013 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:20:05.025 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:20:05.079 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T16:20:05.145 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T16:20:05.183 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T16:20:05.208 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T16:20:05.268 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T16:20:05.597 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T16:20:05.651 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:20:05.773 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:20:05.793 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:20:05.868 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:20:05.906 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T16:20:05.949 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T16:20:06.138 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T16:20:06.189 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T16:20:06.218 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:20:06.323 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:20:06.352 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:20:06.401 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:20:06.440 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T16:20:06.462 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:20:06.478 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:20:06.612 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:20:06.644 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:20:06.683 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T16:20:06.860 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T16:20:06.993 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:20:07.037 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:20:07.136 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:20:07.207 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:20:07.298 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-04-02T16:20:07.375 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T16:20:07.417 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T16:20:07.421 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:20:07.456 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:20:07.465 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T16:20:07.521 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T16:20:07.529 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T16:20:07.586 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T16:20:07.623 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T16:20:07.626 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T16:20:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T16:20:07.654 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T16:20:07.682 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T16:20:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T16:20:07.725 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T16:20:07.798 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T16:20:07.852 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T16:20:07.891 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T16:20:07.954 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T16:20:07.991 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T16:20:08.034 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T16:20:08.067 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T16:20:08.094 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T16:20:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T16:20:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:20:08.163 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T16:20:08.200 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T16:20:08.243 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T16:20:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:20:08.271 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T16:20:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:20:08.340 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T16:20:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:20:08.386 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T16:20:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T16:20:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T16:20:08.459 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T16:20:08.592 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T16:20:08.638 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T16:20:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T16:20:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T16:20:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:20:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:20:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:20:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T16:20:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T16:20:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:20:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T16:20:09.155 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T16:20:09.227 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:20:09.233 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T16:20:09.268 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T16:20:09.305 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T16:20:09.320 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T16:20:09.379 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:20:09.388 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:20:09.426 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T16:20:09.476 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T16:20:09.483 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T16:20:09.514 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T16:20:09.546 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T16:20:09.624 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:20:09.684 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T16:20:09.782 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:20:09.836 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:20:09.839 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T16:20:10.010 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T16:20:10.068 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T16:20:10.097 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T16:20:10.141 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T16:20:10.196 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T16:20:10.286 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T16:20:10.298 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T16:20:10.342 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T16:20:10.344 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T16:20:10.368 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T16:20:10.400 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T16:20:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T16:20:10.471 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T16:20:10.491 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T16:20:10.503 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T16:20:10.539 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T16:20:10.549 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T16:20:10.587 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T16:20:10.594 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T16:20:10.631 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T16:20:10.661 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T16:20:10.674 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T16:20:10.694 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T16:20:10.716 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T16:20:10.738 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T16:20:10.771 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T16:20:10.797 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T16:20:10.849 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T16:20:10.870 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T16:20:10.897 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T16:20:10.903 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T16:20:10.927 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T16:20:10.938 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T16:20:10.966 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T16:20:10.968 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T16:20:11.049 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T16:20:11.062 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T16:20:11.090 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T16:20:11.093 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T16:20:11.134 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T16:20:11.160 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T16:20:11.182 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T16:20:11.209 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T16:20:11.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T16:20:11.321 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T16:20:11.377 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T16:20:11.410 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T16:20:11.525 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T16:20:11.567 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T16:20:11.624 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T16:20:11.682 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T16:20:11.881 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T16:20:11.922 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T16:20:11.937 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T16:20:12.015 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T16:20:12.057 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T16:20:12.095 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T16:20:12.107 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T16:20:12.127 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T16:20:12.148 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:20:12.157 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T16:20:12.166 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T16:20:12.194 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T16:20:12.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T16:20:12.230 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T16:20:12.237 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T16:20:12.268 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T16:20:12.276 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T16:20:12.289 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T16:20:12.333 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T16:20:12.363 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T16:20:12.438 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T16:20:12.473 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T16:20:12.507 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T16:20:12.548 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T16:20:12.578 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T16:20:12.591 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T16:20:12.680 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T16:20:12.761 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T16:20:12.769 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T16:20:12.814 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T16:20:12.845 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T16:20:12.892 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T16:20:12.908 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T16:20:12.989 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T16:20:13.047 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T16:20:13.096 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T16:20:13.239 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T16:20:13.297 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T16:20:13.353 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T16:20:13.427 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T16:20:13.464 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T16:20:13.492 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T16:20:13.540 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T16:20:13.583 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T16:20:13.627 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T16:20:13.669 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T16:20:13.763 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T16:20:13.810 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T16:20:13.816 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T16:20:13.844 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T16:20:13.866 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T16:20:13.892 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T16:20:13.895 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T16:20:13.938 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T16:20:13.973 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T16:20:14.012 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T16:20:14.036 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T16:20:14.071 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T16:20:14.111 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T16:20:14.136 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T16:20:14.161 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T16:20:14.190 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T16:20:14.218 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T16:20:14.232 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T16:20:14.299 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T16:20:14.360 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T16:20:14.472 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T16:20:14.511 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T16:20:14.560 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T16:20:14.625 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T16:20:14.817 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T16:20:14.875 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T16:20:14.929 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T16:20:15.023 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T16:20:15.052 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T16:20:15.064 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T16:20:15.095 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T16:20:15.126 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T16:20:15.163 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T16:20:15.197 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T16:20:15.236 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T16:20:15.268 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T16:20:15.297 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T16:20:15.376 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T16:20:15.408 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T16:20:15.441 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T16:20:15.474 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T16:20:15.509 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T16:20:15.600 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T16:20:15.690 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T16:20:15.732 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T16:20:15.772 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T16:20:15.818 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T16:20:15.919 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T16:20:15.973 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T16:20:16.258 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T16:20:16.332 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T16:20:16.362 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T16:20:16.394 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T16:20:16.422 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T16:20:16.733 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T16:20:16.734 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T16:20:16.760 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:20:16.787 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T16:20:16.821 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T16:20:16.834 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:20:16.865 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T16:20:16.870 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T16:20:16.901 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T16:20:16.959 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T16:20:17.022 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T16:20:17.198 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T16:20:17.228 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T16:20:17.308 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T16:20:17.433 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T16:20:17.546 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:20:17.864 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:20:17.973 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T16:20:18.031 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T16:20:18.086 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T16:20:19.372 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T16:20:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T16:20:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T16:20:19.516 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T16:20:19.543 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T16:20:19.877 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T16:20:19.906 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:20:19.983 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T16:20:20.011 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T16:20:20.048 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T16:20:20.237 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:20:20.286 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:20:20.532 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T16:20:20.560 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:20:20.614 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T16:20:20.636 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T16:20:20.838 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:20:20.873 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:20:20.962 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:20:20.962 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T16:20:20.962 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T16:20:20.962 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:21.000 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:20:22.984 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:20:23.016 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:20:23.591 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T16:20:23.627 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:20:23.709 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T16:20:23.709 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T16:20:23.709 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T16:20:23.709 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:23.746 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:20:33.009 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:20:33.009 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-04-02T16:20:33.009 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-04-02T16:20:33.009 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-04-02T16:20:33.009 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-04-02T16:20:33.009 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-04-02T16:20:33.010 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-04-02T16:20:33.010 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-04-02T16:20:33.010 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:33.172 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:20:33.188 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:20:33.188 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:33.188 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T16:20:33.188 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:34.692 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:20:34.718 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:20:34.719 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:34.719 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T16:20:34.719 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:20:34.719 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:20:34.719 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:34.943 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:20:34.966 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:20:34.966 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:34.966 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T16:20:34.967 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:20:34.967 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:20:34.967 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:35.402 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:20:35.425 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:20:35.425 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:35.425 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T16:20:35.425 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:20:35.425 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:20:35.425 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:35.521 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T16:20:35.564 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T16:20:35.564 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-04-02T16:20:35.565 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-04-02T16:20:35.565 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-04-02T16:20:35.565 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-04-02T16:20:35.565 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-04-02T16:20:35.565 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-04-02T16:20:35.565 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-04-02T16:20:35.565 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:35.732 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:20:35.747 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T16:20:35.747 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:35.748 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T16:20:35.748 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:35.873 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:20:35.903 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:20:35.951 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T16:20:36.012 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T16:20:36.034 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:20:37.060 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:20:37.102 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:20:37.295 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:20:37.320 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T16:20:37.320 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:37.320 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T16:20:37.320 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:20:37.320 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T16:20:37.321 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:37.554 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:20:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T16:20:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T16:20:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:20:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T16:20:37.576 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:20:38.060 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:20:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T16:20:38.068 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:38.068 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T16:20:38.068 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:20:38.068 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T16:20:38.068 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:38.094 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:20:38.165 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T16:20:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:20:38.348 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T16:20:38.372 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:20:38.395 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:20:38.395 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:38.395 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T16:20:38.395 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:20:38.395 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:20:38.395 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:38.439 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:20:38.451 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:20:38.506 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T16:20:38.601 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:20:38.630 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T16:20:38.663 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T16:20:38.725 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T16:20:38.753 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:20:39.380 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T16:20:39.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:20:39.431 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:20:39.431 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:39.431 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T16:20:39.431 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:20:39.431 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:20:39.432 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:39.475 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:20:39.497 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:20:39.497 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:39.497 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T16:20:39.497 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:39.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:20:39.565 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:20:39.565 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:39.565 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T16:20:39.565 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:20:39.565 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:20:39.565 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:39.839 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T16:20:39.868 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:20:39.871 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:20:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:20:39.890 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:39.890 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T16:20:39.890 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:20:39.891 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:20:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:40.883 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T16:20:40.906 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:20:41.014 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T16:20:41.175 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T16:20:41.208 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:20:41.230 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T16:20:41.230 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:41.230 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T16:20:41.230 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:20:41.230 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T16:20:41.230 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:41.301 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:20:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T16:20:41.366 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T16:20:42.283 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T16:20:42.310 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:20:42.332 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T16:20:42.333 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:42.333 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T16:20:42.333 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:20:42.333 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:20:42.333 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:42.377 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:20:42.399 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T16:20:42.399 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:42.399 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T16:20:42.399 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:42.445 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:20:42.466 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:20:42.467 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:42.467 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T16:20:42.467 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:20:42.467 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:20:42.467 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:42.787 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:20:42.810 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T16:20:42.810 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T16:20:42.810 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T16:20:42.810 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:20:42.810 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T16:20:42.810 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:20:44.348 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T16:20:44.389 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T16:20:44.419 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T16:20:44.447 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T16:20:44.628 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T16:20:44.746 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T16:20:44.795 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T16:20:44.841 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T16:20:44.887 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T16:20:44.922 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T16:20:44.955 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T16:20:44.995 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T16:20:45.025 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T16:20:45.096 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T16:20:45.168 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T16:20:45.207 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T16:20:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:20:45.883 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:20:45.902 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:20:46.709 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T16:20:46.794 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T16:20:46.835 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T16:20:46.922 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T16:20:46.956 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T16:20:46.989 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T16:20:47.015 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T16:20:47.041 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T16:20:47.042 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T16:20:47.121 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T16:20:47.150 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T16:20:47.151 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T16:20:47.236 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T16:20:47.262 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T16:20:47.295 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T16:20:47.320 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T16:20:47.345 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T16:20:47.379 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T16:20:47.418 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T16:20:47.450 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T16:20:47.476 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T16:20:47.501 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T16:20:47.526 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T16:20:47.536 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T16:20:47.551 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T16:20:47.558 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:20:47.558 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:20:47.586 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T16:20:47.616 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T16:20:47.643 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T16:20:47.746 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:20:47.746 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T16:20:47.820 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T16:20:47.846 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T16:20:47.852 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T16:20:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T16:20:47.905 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T16:20:47.930 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T16:20:47.963 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T16:20:47.995 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T16:20:48.008 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T16:20:48.057 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T16:20:48.107 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T16:20:48.142 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T16:20:48.196 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T16:20:48.224 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T16:20:48.253 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T16:20:48.312 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T16:20:48.407 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T16:20:48.443 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T16:20:49.070 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:20:49.108 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:20:49.128 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T16:20:50.008 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T16:20:50.095 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T16:20:50.129 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T16:20:50.233 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T16:20:50.273 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T16:20:50.298 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T16:20:50.324 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T16:20:50.350 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T16:20:50.350 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T16:20:50.422 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T16:20:50.451 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T16:20:50.451 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T16:20:50.536 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T16:20:50.579 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T16:20:50.621 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T16:20:50.646 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T16:20:50.671 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T16:20:50.705 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T16:20:50.752 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T16:20:50.792 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T16:20:50.818 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T16:20:50.842 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T16:20:50.868 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T16:20:50.893 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T16:20:50.900 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:20:50.900 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:20:51.096 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T16:20:51.096 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T16:20:51.187 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T16:20:51.219 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T16:20:51.246 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T16:20:51.272 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T16:20:51.296 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T16:20:51.365 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T16:20:53.463 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T16:20:53.463 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T16:20:53.463 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:20:53.464 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T16:20:53.465 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T16:20:53.466 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T16:20:53.466 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T16:20:53.466 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T16:20:53.466 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T16:20:53.466 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T16:20:53.466 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T16:20:53.466 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T16:20:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T16:20:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T16:20:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T16:20:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T16:20:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T16:20:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T16:20:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T16:20:53.472 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T16:20:53.473 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T16:20:53.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T16:20:53.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T16:20:53.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T16:20:53.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T16:20:53.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T16:20:53.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T16:20:53.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T16:20:53.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T16:20:53.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T16:20:53.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T16:20:53.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T16:20:53.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T16:20:53.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T16:20:53.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T16:20:53.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T16:20:56.389 INFO:teuthology.orchestra.run.smithi188.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.390 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.391 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.392 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T16:20:56.393 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-16.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.394 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T16:20:56.395 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T16:20:56.396 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T16:20:56.397 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T16:20:56.398 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T16:20:56.399 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.400 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T16:20:56.401 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:20:56.402 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-02T16:20:56.776 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T16:20:56.776 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T16:20:56.776 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T16:20:56.776 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T16:20:56.776 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T16:20:56.777 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T16:20:56.778 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T16:20:56.796 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T16:20:56.797 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T16:20:56.797 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T16:20:56.797 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T16:20:56.797 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T16:20:56.797 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T16:20:56.797 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T16:20:56.797 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T16:20:56.798 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T16:20:56.798 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T16:20:56.798 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T16:20:56.798 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T16:20:56.798 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T16:20:56.798 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T16:20:56.798 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T16:20:56.799 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T16:20:56.799 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T16:20:56.799 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T16:20:56.799 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T16:20:56.799 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T16:20:56.799 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T16:20:56.800 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T16:20:56.800 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T16:20:56.800 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T16:20:56.800 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T16:20:56.800 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T16:20:56.800 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T16:20:56.800 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T16:20:56.801 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T16:20:56.801 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T16:20:56.801 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T16:20:56.801 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T16:20:56.801 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T16:20:56.802 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T16:20:56.803 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T16:20:56.804 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T16:20:56.805 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T16:20:56.806 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T16:20:56.807 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T16:20:56.808 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T16:20:56.809 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T16:20:56.810 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T16:20:56.811 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T16:20:56.812 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T16:20:56.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T16:20:56.814 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T16:20:56.814 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T16:20:56.877 DEBUG:teuthology.parallel:result is None 2024-04-02T16:21:00.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T16:21:00.061 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:00.061 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-04-02T16:21:00.061 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T16:21:00.061 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T16:21:00.061 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.062 INFO:teuthology.orchestra.run.smithi050.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.063 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.064 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.064 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.064 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.064 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.064 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.064 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.065 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.065 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.065 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.065 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.065 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.065 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.065 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.066 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.066 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.066 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.066 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.066 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.066 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.067 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.067 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.067 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T16:21:00.067 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T16:21:00.067 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T16:21:00.068 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T16:21:00.068 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T16:21:00.068 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T16:21:00.068 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T16:21:00.068 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T16:21:00.068 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T16:21:00.068 INFO:teuthology.orchestra.run.smithi050.stdout: jq-1.6-16.el9.x86_64 2024-04-02T16:21:00.069 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T16:21:00.069 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T16:21:00.069 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T16:21:00.069 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.069 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.069 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.070 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T16:21:00.070 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T16:21:00.070 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T16:21:00.070 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T16:21:00.070 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T16:21:00.070 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.070 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.071 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T16:21:00.071 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.071 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T16:21:00.071 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T16:21:00.071 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T16:21:00.071 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T16:21:00.072 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T16:21:00.072 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T16:21:00.072 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T16:21:00.072 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T16:21:00.072 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T16:21:00.072 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T16:21:00.073 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T16:21:00.073 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T16:21:00.073 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T16:21:00.073 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T16:21:00.073 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T16:21:00.073 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T16:21:00.073 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T16:21:00.074 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T16:21:00.074 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T16:21:00.074 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T16:21:00.074 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T16:21:00.074 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T16:21:00.074 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T16:21:00.074 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T16:21:00.075 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.075 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.075 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.075 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T16:21:00.075 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T16:21:00.076 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T16:21:00.076 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T16:21:00.076 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T16:21:00.076 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T16:21:00.076 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T16:21:00.076 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T16:21:00.077 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T16:21:00.077 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T16:21:00.077 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T16:21:00.077 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T16:21:00.077 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T16:21:00.077 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T16:21:00.077 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T16:21:00.078 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T16:21:00.078 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T16:21:00.078 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T16:21:00.078 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T16:21:00.078 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T16:21:00.078 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T16:21:00.079 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T16:21:00.079 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T16:21:00.079 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T16:21:00.079 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T16:21:00.079 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T16:21:00.079 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T16:21:00.079 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T16:21:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T16:21:00.081 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T16:21:00.082 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T16:21:00.083 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T16:21:00.083 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T16:21:00.083 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:00.083 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-04-02T16:21:00.496 DEBUG:teuthology.parallel:result is None 2024-04-02T16:21:00.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:21:00.647 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T16:21:00.669 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T16:21:00.670 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T16:21:00.670 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T16:21:00.672 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:21:00.820 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T16:21:00.846 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T16:21:00.846 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T16:21:00.847 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T16:21:00.848 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T16:21:00.849 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:00.849 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T16:21:00.878 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:00.878 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T16:21:00.920 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T16:21:00.921 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:00.921 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T16:21:00.948 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T16:21:01.015 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:01.015 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T16:21:01.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T16:21:01.115 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T16:21:01.115 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:01.115 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T16:21:01.141 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T16:21:01.208 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:01.208 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T16:21:01.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T16:21:01.307 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T16:21:01.307 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:01.308 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T16:21:01.334 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T16:21:01.402 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:01.403 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T16:21:01.430 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T16:21:01.498 INFO:teuthology.run_tasks:Running task ceph... 2024-04-02T16:21:01.594 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-02T16:21:01.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /var/log/ceph 2024-04-02T16:21:01.597 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-04-02T16:21:01.623 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-02T16:21:01.624 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-02T16:21:01.663 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-02T16:21:01.690 INFO:tasks.ceph:Creating extra log directories... 2024-04-02T16:21:01.690 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-02T16:21:01.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-02T16:21:01.763 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-02T16:21:01.764 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-02T16:21:01.764 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-02T16:21:01.764 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:01.764 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-02T16:21:01.809 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-02T16:21:01.925 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-02T16:21:01.952 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:01.952 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-02T16:21:01.968 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-02T16:21:02.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-02T16:21:02.125 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-02T16:21:02.126 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-02T16:21:02.126 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635705', 'branch': 'wip-batrick-testing-20240402.004512', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7635705', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240402.004512'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240402.004512', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2763, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'sleep_before_teardown': 0, 'subset': '318/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240402.004512', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi050.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNbSUWgiMdsDSYGPx6UsoJ1GP0ucxTqSTLsMFQwRZa1BJ31hojCBoDUJx6g0VweD55pl+tav2SeHhVtPxYFlaYw=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFcNDB6+8zz3+sp8lJHB7E/fUtr0BSP2sNheFVRQ/v+J/YulZRb0hF57mceu+lJ4IqwuCl+jC+jOynvD7bdbGfY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-04-02_11:52:43', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-02T16:21:02.127 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-02T16:21:02.130 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-02T16:21:02.170 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2024-04-02T16:21:02.184 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-04-02T16:21:02.236 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:02.237 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T16:21:02.262 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T16:21:02.262 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-04-02T16:21:02.317 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T16:21:02.317 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:02.317 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-02T16:21:02.317 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:02.317 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:02.318 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 16:20:51.709908130 +0000 2024-04-02T16:21:02.318 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 16:18:39.415519343 +0000 2024-04-02T16:21:02.318 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 16:18:39.415519343 +0000 2024-04-02T16:21:02.318 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 16:18:39.415519343 +0000 2024-04-02T16:21:02.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T16:21:02.383 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T16:21:02.383 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T16:21:02.383 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 8.2206e-05 s, 6.2 MB/s 2024-04-02T16:21:02.384 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T16:21:02.440 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 16:20:51.709908130 +0000 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 16:18:39.959505801 +0000 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 16:18:39.959505801 +0000 2024-04-02T16:21:02.495 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 16:18:39.959505801 +0000 2024-04-02T16:21:02.495 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T16:21:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T16:21:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T16:21:02.560 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.8768e-05 s, 5.2 MB/s 2024-04-02T16:21:02.561 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T16:21:02.616 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 16:20:51.710908105 +0000 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 16:18:40.296497413 +0000 2024-04-02T16:21:02.671 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 16:18:40.296497413 +0000 2024-04-02T16:21:02.672 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 16:18:40.296497413 +0000 2024-04-02T16:21:02.672 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T16:21:02.736 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T16:21:02.736 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T16:21:02.736 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 7.985e-05 s, 6.4 MB/s 2024-04-02T16:21:02.737 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T16:21:02.793 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-04-02T16:21:02.847 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T16:21:02.847 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:02.847 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-02T16:21:02.848 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:02.848 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:02.848 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 16:20:51.710908105 +0000 2024-04-02T16:21:02.848 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 16:18:40.622489298 +0000 2024-04-02T16:21:02.848 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 16:18:40.622489298 +0000 2024-04-02T16:21:02.848 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 16:18:40.622489298 +0000 2024-04-02T16:21:02.848 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T16:21:02.914 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T16:21:02.914 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T16:21:02.914 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.5157e-05 s, 5.4 MB/s 2024-04-02T16:21:02.915 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T16:21:02.971 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:02.971 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:02.971 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T16:21:02.988 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T16:21:02.988 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-04-02T16:21:03.044 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 16:20:48.326799693 +0000 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 16:18:48.486589153 +0000 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 16:18:48.486589153 +0000 2024-04-02T16:21:03.045 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 16:18:48.486589153 +0000 2024-04-02T16:21:03.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T16:21:03.112 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T16:21:03.112 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T16:21:03.112 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000284641 s, 1.8 MB/s 2024-04-02T16:21:03.113 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T16:21:03.170 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-04-02T16:21:03.226 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T16:21:03.226 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:03.226 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-04-02T16:21:03.226 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:03.227 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:03.227 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 16:20:48.327799676 +0000 2024-04-02T16:21:03.227 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 16:18:48.871582053 +0000 2024-04-02T16:21:03.227 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 16:18:48.871582053 +0000 2024-04-02T16:21:03.227 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 16:18:48.871582053 +0000 2024-04-02T16:21:03.227 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T16:21:03.293 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T16:21:03.293 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T16:21:03.294 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000218444 s, 2.3 MB/s 2024-04-02T16:21:03.296 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T16:21:03.351 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 16:20:48.327799676 +0000 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 16:18:49.207575857 +0000 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 16:18:49.207575857 +0000 2024-04-02T16:21:03.407 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 16:18:49.207575857 +0000 2024-04-02T16:21:03.408 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T16:21:03.472 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T16:21:03.473 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T16:21:03.473 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000190065 s, 2.7 MB/s 2024-04-02T16:21:03.474 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T16:21:03.530 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-04-02T16:21:03.584 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T16:21:03.584 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:21:03.585 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-02T16:21:03.585 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:21:03.585 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T16:21:03.585 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 16:20:48.327799676 +0000 2024-04-02T16:21:03.585 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 16:18:49.539569734 +0000 2024-04-02T16:21:03.585 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 16:18:49.539569734 +0000 2024-04-02T16:21:03.585 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 16:18:49.539569734 +0000 2024-04-02T16:21:03.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T16:21:03.650 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T16:21:03.651 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T16:21:03.651 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000164027 s, 3.1 MB/s 2024-04-02T16:21:03.652 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T16:21:03.710 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:03.710 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-02T16:21:03.710 INFO:tasks.ceph:Generating config... 2024-04-02T16:21:03.711 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-02T16:21:03.711 INFO:tasks.ceph:[client] debug client = 20 2024-04-02T16:21:03.711 INFO:tasks.ceph:[client] debug ms = 1 2024-04-02T16:21:03.711 INFO:tasks.ceph:[client] pid file = 2024-04-02T16:21:03.712 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-02T16:21:03.712 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-02T16:21:03.712 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-02T16:21:03.712 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-02T16:21:03.713 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-02T16:21:03.713 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-02T16:21:03.713 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-02T16:21:03.713 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-02T16:21:03.713 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-02T16:21:03.713 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-02T16:21:03.713 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-02T16:21:03.714 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-02T16:21:03.714 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-02T16:21:03.714 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-02T16:21:03.714 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-02T16:21:03.714 INFO:tasks.ceph:Setting up mon.a... 2024-04-02T16:21:03.714 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-02T16:21:03.765 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.keyring 2024-04-02T16:21:03.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-02T16:21:03.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T16:21:03.841 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.50'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2024-04-02T16:21:03.841 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.50,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-02T16:21:03.842 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:03.842 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-02T16:21:03.896 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.50 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:03.982 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: generated fsid 057e6c06-7c98-48a0-9312-9e11e9abe7fe 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:setting min_mon_release = quincy 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 0 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:fsid 057e6c06-7c98-48a0-9312-9e11e9abe7fe 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:last_changed 2024-04-02T16:21:03.983028+0000 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-04-02T16:21:03.983028+0000 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:min_mon_release 17 (quincy) 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:election_strategy: 1 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2024-04-02T16:21:03.983 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-02T16:21:03.985 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-02T16:21:03.999 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 057e6c06-7c98-48a0-9312-9e11e9abe7fe... 2024-04-02T16:21:04.000 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-02T16:21:04.042 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-02T16:21:04.090 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2024-04-02T16:21:04.090 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2024-04-02T16:21:04.090 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2024-04-02T16:21:04.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: # disable pg_autoscaler by default for new pools 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: mon_allow_pool_size_one = true 2024-04-02T16:21:04.092 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: mon health detail to clog = false 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = "172.21.15.50,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on skipped message = False 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 057e6c06-7c98-48a0-9312-9e11e9abe7fe 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2024-04-02T16:21:04.093 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock profile = high_recovery_ops 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug trim objects = true 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2024-04-02T16:21:04.094 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore block size = 96636764160 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore fsck on mount = True 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluefs = 20 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluestore = 20 2024-04-02T16:21:04.095 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 20 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: debug rocksdb = 10 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd backfillfull_ratio = 0.85 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd full ratio = 0.9 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd nearfull ratio = 0.8 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: osd failsafe full ratio = 0.95 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: osd objectstore = bluestore 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:21:04.096 INFO:teuthology.orchestra.run.smithi050.stdout: mgr/telemetry/nag = false 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: debug client = 20 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-02T16:21:04.097 INFO:teuthology.orchestra.run.smithi050.stdout: auth mon ticket ttl = 660 # 11m 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: auth service ticket ttl = 240 # 4m 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: # don't complain about insecure global_id in the test suite 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: # 1m isn't quite enough 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: mon_down_mkfs_grace = 2m 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_filestore_osds = false 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: mon down mkfs grace = 300 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: mon op complaint time = 120 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:04.098 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2024-04-02T16:21:04.099 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2024-04-02T16:21:04.099 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2024-04-02T16:21:04.099 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2024-04-02T16:21:04.099 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-02T16:21:04.099 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-02T16:21:04.099 INFO:teuthology.orchestra.run.smithi050.stdout: client mount timeout = 600 2024-04-02T16:21:04.102 INFO:teuthology.orchestra.run.smithi050.stdout: debug client = 20 2024-04-02T16:21:04.102 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-04-02T16:21:04.102 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = "" 2024-04-02T16:21:04.102 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2024-04-02T16:21:04.102 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2024-04-02T16:21:04.102 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.b] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.c] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout:[mds] 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: debug mds = 20 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: debug mds balancer = 20 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug frag = True 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: mds debug scatterstat = True 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: mds op complaint time = 180 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: mds verify scatter = True 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: osd op complaint time = 180 2024-04-02T16:21:04.103 INFO:teuthology.orchestra.run.smithi050.stdout: rados mon op timeout = 900 2024-04-02T16:21:04.104 INFO:teuthology.orchestra.run.smithi050.stdout: rados osd op timeout = 900 2024-04-02T16:21:04.104 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-02T16:21:04.104 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-02T16:21:04.104 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-02T16:21:04.104 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-02T16:21:04.104 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.105 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-02T16:21:04.106 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.50,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-04-02T16:21:04.107 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 057e6c06-7c98-48a0-9312-9e11e9abe7fe 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-02T16:21:04.108 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-02T16:21:04.109 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-02T16:21:04.110 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-02T16:21:04.111 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-02T16:21:04.112 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: mon down mkfs grace = 300 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-02T16:21:04.113 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-04-02T16:21:04.114 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-02T16:21:04.115 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-02T16:21:04.116 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-02T16:21:04.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-02T16:21:04.211 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-02T16:21:04.212 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:04.212 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-02T16:21:04.226 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:04.226 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-02T16:21:04.280 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:21:04.280 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:04.280 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-02T16:21:04.280 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T16:21:04.361 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:04.361 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:04.415 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:21:04.415 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:04.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-02T16:21:04.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T16:21:04.456 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:04.456 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:04.511 INFO:tasks.ceph:Setting up mon nodes... 2024-04-02T16:21:04.511 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-02T16:21:04.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-02T16:21:04.565 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-02T16:21:04.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-02T16:21:04.630 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-02T16:21:04.633 INFO:tasks.ceph:Setting up mds nodes... 2024-04-02T16:21:04.633 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-02T16:21:04.688 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-02T16:21:04.690 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-02T16:21:04.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-02T16:21:04.811 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-02T16:21:04.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-02T16:21:04.840 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-02T16:21:04.894 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-02T16:21:04.897 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-02T16:21:04.924 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-02T16:21:05.018 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-02T16:21:05.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-02T16:21:05.047 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-02T16:21:05.047 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T16:21:05.090 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-02T16:21:05.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-02T16:21:05.188 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-02T16:21:05.203 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-02T16:21:05.245 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-02T16:21:05.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-02T16:21:05.346 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-02T16:21:05.359 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-02T16:21:05.359 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-02T16:21:05.359 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-02T16:21:05.386 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:05.386 INFO:tasks.ceph:role: osd.0 2024-04-02T16:21:05.386 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:21:05.386 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:05.452 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:05.453 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:05.484 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-04-02T16:21:05.486 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-04-02T16:21:05.486 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-02T16:21:05.557 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-02T16:21:05.629 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-02T16:21:05.697 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:05.697 INFO:tasks.ceph:role: osd.1 2024-04-02T16:21:05.697 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:21:05.697 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-02T16:21:05.763 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:05.764 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:05.796 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-04-02T16:21:05.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-04-02T16:21:05.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-02T16:21:05.869 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-02T16:21:05.940 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-02T16:21:06.007 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:06.007 INFO:tasks.ceph:role: osd.2 2024-04-02T16:21:06.007 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:21:06.007 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-02T16:21:06.074 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:06.074 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:06.075 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:06.107 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-04-02T16:21:06.110 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-04-02T16:21:06.110 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-02T16:21:06.180 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-02T16:21:06.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-02T16:21:06.321 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:06.321 INFO:tasks.ceph:role: osd.3 2024-04-02T16:21:06.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:21:06.321 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-02T16:21:06.387 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:06.388 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:06.420 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-04-02T16:21:06.422 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-04-02T16:21:06.422 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-02T16:21:06.492 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-02T16:21:06.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:06.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:06.662+0000 7f177a648640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-02T16:21:06.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:06.662+0000 7f177a648640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-02T16:21:06.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:06.662+0000 7f177a648640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-02T16:21:06.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:06.662+0000 7f177a648640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-02T16:21:06.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:06.662+0000 7f177a648640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-02T16:21:06.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:06.662+0000 7f177a648640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-02T16:21:09.512 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T16:21:09.544 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:09.633+0000 7feafe1e1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-02T16:21:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:09.634+0000 7feafe1e1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-02T16:21:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:09.634+0000 7feafe1e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-02T16:21:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:09.634+0000 7feafe1e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-02T16:21:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:09.634+0000 7feafe1e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-02T16:21:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:09.634+0000 7feafe1e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-02T16:21:12.479 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T16:21:12.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:12.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:12.607+0000 7f1f798c8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-02T16:21:12.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:12.607+0000 7f1f798c8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-02T16:21:12.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:12.607+0000 7f1f798c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-02T16:21:12.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:12.607+0000 7f1f798c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-02T16:21:12.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:12.608+0000 7f1f798c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-02T16:21:12.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:12.608+0000 7f1f798c8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-02T16:21:15.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T16:21:15.482 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:15.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:15.573+0000 7f6c41611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-02T16:21:15.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:15.573+0000 7f6c41611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-02T16:21:15.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:15.573+0000 7f6c41611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-02T16:21:15.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:15.573+0000 7f6c41611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-02T16:21:15.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:15.573+0000 7f6c41611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-02T16:21:15.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:15.574+0000 7f6c41611640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-02T16:21:18.413 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T16:21:18.445 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-02T16:21:18.477 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:18.477 INFO:tasks.ceph:role: osd.4 2024-04-02T16:21:18.477 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:21:18.477 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-02T16:21:18.555 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:18.555 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:18.555 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:18.555 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:18.555 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:18.556 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:18.556 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:18.556 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:18.556 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:18.556 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:19.063 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-02T16:21:19.065 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-02T16:21:19.066 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-02T16:21:19.172 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-02T16:21:19.209 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-02T16:21:19.278 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:19.279 INFO:tasks.ceph:role: osd.5 2024-04-02T16:21:19.279 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:21:19.279 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-02T16:21:19.353 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:19.353 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:19.353 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:19.353 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:19.354 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:19.354 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:19.354 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:19.354 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:19.354 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:19.354 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:19.812 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-02T16:21:19.814 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-02T16:21:19.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-02T16:21:19.921 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-02T16:21:19.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-02T16:21:20.023 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:20.023 INFO:tasks.ceph:role: osd.6 2024-04-02T16:21:20.023 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:21:20.023 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-02T16:21:20.099 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:20.099 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:20.100 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:20.602 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-02T16:21:20.604 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-02T16:21:20.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-02T16:21:20.724 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-02T16:21:20.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-02T16:21:20.827 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:21:20.828 INFO:tasks.ceph:role: osd.7 2024-04-02T16:21:20.828 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:21:20.828 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-02T16:21:20.903 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:21:20.903 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:21:20.903 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:21:20.903 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T16:21:20.903 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:21:20.903 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:21:20.904 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:21:20.904 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T16:21:20.904 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:21:20.904 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:21:21.400 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-02T16:21:21.403 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-02T16:21:21.403 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-02T16:21:21.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-02T16:21:21.572 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:21.672 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:21.671+0000 7ff1901ac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-02T16:21:21.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:21.672+0000 7ff1901ac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-02T16:21:21.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:21.672+0000 7ff1901ac640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-02T16:21:21.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:21.672+0000 7ff1901ac640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-02T16:21:21.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:21.672+0000 7ff1901ac640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-02T16:21:21.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:21.672+0000 7ff1901ac640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-02T16:21:24.519 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T16:21:24.550 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:24.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:24.639+0000 7f0fb720d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-02T16:21:24.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:24.640+0000 7f0fb720d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-02T16:21:24.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:24.640+0000 7f0fb720d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-02T16:21:24.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:24.640+0000 7f0fb720d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-02T16:21:24.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:24.640+0000 7f0fb720d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-02T16:21:24.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:24.640+0000 7f0fb720d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-02T16:21:27.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T16:21:27.513 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:27.605 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:27.604+0000 7f4675c11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-02T16:21:27.605 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:27.604+0000 7f4675c11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-02T16:21:27.606 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:27.604+0000 7f4675c11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-02T16:21:27.606 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:27.604+0000 7f4675c11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-02T16:21:27.606 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:27.604+0000 7f4675c11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-02T16:21:27.606 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:27.605+0000 7f4675c11640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-02T16:21:30.448 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-02T16:21:30.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:30.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:30.567+0000 7f201f773640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-02T16:21:30.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:30.568+0000 7f201f773640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-02T16:21:30.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:30.568+0000 7f201f773640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-02T16:21:30.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:30.568+0000 7f201f773640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-02T16:21:30.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:30.568+0000 7f201f773640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-02T16:21:30.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:30.568+0000 7f201f773640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-02T16:21:32.128 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:21:32.131 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:21:33.415 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-02T16:21:33.444 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-02T16:21:33.444 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:33.444 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-02T16:21:33.475 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:33.476 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-02T16:21:33.546 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:33.546 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-02T16:21:33.618 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:33.618 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-02T16:21:33.685 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:33.685 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-02T16:21:33.754 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:33.754 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-02T16:21:33.823 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:33.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-02T16:21:33.891 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:33.891 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-02T16:21:33.922 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:33.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-02T16:21:33.992 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:33.992 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-02T16:21:34.061 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:34.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-02T16:21:34.129 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:34.130 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-02T16:21:34.197 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:34.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-02T16:21:34.265 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:34.265 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-02T16:21:34.333 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:34.333 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-02T16:21:34.351 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:34.351 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-02T16:21:34.408 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:34.409 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-02T16:21:34.424 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:34.424 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-02T16:21:34.480 INFO:tasks.ceph:Adding keys to all mons... 2024-04-02T16:21:34.480 DEBUG:teuthology.orchestra.run.smithi050:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-02T16:21:34.482 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.y] 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBwMAxmxWS5IRAAfhIqHEPQPduTKpEb/FzWyQ== 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a] 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBwMAxm+V4IKRAAfLlvRXFY4am6ultvJitXhA== 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c] 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBwMAxmjdFkMBAA43Cl690AwszHSmVuV6K34A== 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQByMAxm+ouKJxAAcQqKvQxmC4w0l6fMLRdJSA== 2024-04-02T16:21:34.523 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQB1MAxm0OfbJRAAgeRoJsctcMiE34ZL6w8gew== 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQB4MAxmwHNOJBAAqW6k+GzchpBdqZr/IRSeBA== 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.3] 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQB7MAxmHAM3IhAAXuRf8UC3Su9y1y5ftgkLmQ== 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.x] 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBwMAxm8qOcJRAAXFEN1C5WhcHJPlmONrBFPA== 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b] 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBwMAxmhcJaNRAAk6QDmBeJUzVdEnJ0um5B+w== 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d] 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBxMAxmTkIeARAAPBkXSr4GSRcEvRwKJqPccw== 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.4] 2024-04-02T16:21:34.524 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCBMAxmC3gaKBAASt1YEuCSsfpokgjPWVWzaQ== 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.5] 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCEMAxmKQg0JhAATb/qM58zw+QbS+oB63E/bQ== 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.6] 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCHMAxm6RoVJBAAUmYck7HG8K5l18FjFfiFRw== 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.7] 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCKMAxm11rnIRAApt58el/7EFrhuffEjIenWg== 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBxMAxmDpViBRAAI+QEeaTzgk7SzJG7sathbQ== 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout:[client.2] 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBxMAxmO3A7CxAA7rH8JU08Yt6Soho9oaGLtg== 2024-04-02T16:21:34.525 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2024-04-02T16:21:34.526 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBxMAxm+7amDhAACmXb4dph7u9ZFU0LXyDCzQ== 2024-04-02T16:21:34.526 INFO:teuthology.orchestra.run.smithi050.stdout:[client.3] 2024-04-02T16:21:34.526 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBxMAxmuP2HFBAAjiZki3fyRvJtJ4mO6vn+fw== 2024-04-02T16:21:34.546 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-04-02T16:21:34.546 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwMAxmxWS5IRAAfhIqHEPQPduTKpEb/FzWyQ== 2024-04-02T16:21:34.546 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwMAxm+V4IKRAAfLlvRXFY4am6ultvJitXhA== 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwMAxmjdFkMBAA43Cl690AwszHSmVuV6K34A== 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQByMAxm+ouKJxAAcQqKvQxmC4w0l6fMLRdJSA== 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQB1MAxm0OfbJRAAgeRoJsctcMiE34ZL6w8gew== 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQB4MAxmwHNOJBAAqW6k+GzchpBdqZr/IRSeBA== 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQB7MAxmHAM3IhAAXuRf8UC3Su9y1y5ftgkLmQ== 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwMAxm8qOcJRAAXFEN1C5WhcHJPlmONrBFPA== 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-02T16:21:34.547 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwMAxmhcJaNRAAk6QDmBeJUzVdEnJ0um5B+w== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBxMAxmTkIeARAAPBkXSr4GSRcEvRwKJqPccw== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCBMAxmC3gaKBAASt1YEuCSsfpokgjPWVWzaQ== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCEMAxmKQg0JhAATb/qM58zw+QbS+oB63E/bQ== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCHMAxm6RoVJBAAUmYck7HG8K5l18FjFfiFRw== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCKMAxm11rnIRAApt58el/7EFrhuffEjIenWg== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBxMAxmDpViBRAAI+QEeaTzgk7SzJG7sathbQ== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBxMAxmO3A7CxAA7rH8JU08Yt6Soho9oaGLtg== 2024-04-02T16:21:34.548 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-04-02T16:21:34.549 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBxMAxm+7amDhAACmXb4dph7u9ZFU0LXyDCzQ== 2024-04-02T16:21:34.549 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2024-04-02T16:21:34.549 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBxMAxmuP2HFBAAjiZki3fyRvJtJ4mO6vn+fw== 2024-04-02T16:21:34.550 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:21:34.571 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:21:34.642 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:34.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:34.700 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:34.703 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:34.759 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.761 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.812 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.867 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.869 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.921 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:34.975 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:21:34.977 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:21:35.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:35.032 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:35.087 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:35.089 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:21:35.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.199 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.202 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.309 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.311 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:21:35.363 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.365 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.421 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.423 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.478 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.536 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.538 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:21:35.594 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-02T16:21:35.595 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-02T16:21:35.622 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-02T16:21:35.719 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-02T16:21:35.745 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-02T16:21:35.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-02T16:21:35.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-02T16:21:35.897 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-02T16:21:35.964 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-02T16:21:36.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-02T16:21:36.087 DEBUG:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:36.089 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:21:36.142 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-02T16:21:36.142 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-02T16:21:36.143 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-02T16:21:36.145 INFO:tasks.ceph.mon.a:Started 2024-04-02T16:21:36.145 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-02T16:21:36.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-02T16:21:36.185 INFO:tasks.ceph.mon.b:Started 2024-04-02T16:21:36.185 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-02T16:21:36.185 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-02T16:21:36.187 INFO:tasks.ceph.mon.c:Started 2024-04-02T16:21:36.187 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-02T16:21:36.188 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-02T16:21:36.188 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-02T16:21:36.190 INFO:tasks.ceph.mgr.y:Started 2024-04-02T16:21:36.190 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-02T16:21:36.190 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-02T16:21:36.192 INFO:tasks.ceph.mgr.x:Started 2024-04-02T16:21:36.192 INFO:tasks.ceph:Setting crush tunables to default 2024-04-02T16:21:36.192 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd crush tunables default 2024-04-02T16:21:36.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.292+0000 7fc157070640 1 Processor -- start 2024-04-02T16:21:36.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.292+0000 7fc157070640 1 -- start start 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc157070640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 0x7fc1500b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc157070640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 0x7fc1500cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc157070640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1500ce110 con 0x7fc15015c340 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc157070640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fc1500d0710 con 0x7fc15015c720 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc157070640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc1500cd030 con 0x7fc15006f630 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc14ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 0x7fc1500b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc154e0c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc15015c340 0x7fc1500cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:39960/0 (socket says 172.21.15.50:39960) 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc154e0c640 1 -- 172.21.15.50:0/2048326331 learned_addr learned my addr 172.21.15.50:0/2048326331 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:36.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc14ffff640 1 -- 172.21.15.50:0/2048326331 >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 msgr2=0x7fc1500b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.293+0000 7fc14ffff640 1 -- 172.21.15.50:0/2048326331 >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 msgr2=0x7fc1500b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.294+0000 7fc14ffff640 1 --2- 172.21.15.50:0/2048326331 >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 0x7fc1500b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.294+0000 7fc14ffff640 1 --2- 172.21.15.50:0/2048326331 >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 0x7fc1500b4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.294+0000 7fc15560d640 1 --2- 172.21.15.50:0/2048326331 >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 0x7fc1500cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.294+0000 7fc15560d640 1 -- 172.21.15.50:0/2048326331 >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 msgr2=0x7fc1500cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.294+0000 7fc15560d640 1 -- 172.21.15.50:0/2048326331 >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 msgr2=0x7fc1500cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.294+0000 7fc15560d640 1 --2- 172.21.15.50:0/2048326331 >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 0x7fc1500cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T16:21:36.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.294+0000 7fc15560d640 1 --2- 172.21.15.50:0/2048326331 >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 0x7fc1500cdbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542685302 0 0) 0x7fc148001490 con 0x7fc15015c340 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc138003520 con 0x7fc15015c340 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 288515851 0 0) 0x7fc138003520 con 0x7fc15015c340 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 msgr2=0x7fc1500cdbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 --2- 172.21.15.50:0/2048326331 >> v2:172.21.15.188:3300/0 conn(0x7fc15006f630 0x7fc1500cdbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 msgr2=0x7fc1500b4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 --2- 172.21.15.50:0/2048326331 >> v2:172.21.15.50:3300/0 conn(0x7fc15015c720 0x7fc1500b4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1500d0960 con 0x7fc15015c340 2024-04-02T16:21:36.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fc148001bc0 con 0x7fc15015c340 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 >> v1:172.21.15.50:6789/0 conn(0x7fc15015c340 legacy=0x7fc1500cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1380039b0 0x7fc138003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1380051c0 0x7fc13800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500d0710 con 0x7fc138004800 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500cd030 con 0x7fc1380039b0 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc13800c0c0 con 0x7fc1380051c0 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14ffff640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc15560d640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1380051c0 0x7fc13800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc154e0c640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1380039b0 0x7fc138003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14ffff640 1 -- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1380051c0 msgr2=0x7fc13800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14ffff640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1380051c0 0x7fc13800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14ffff640 1 -- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1380039b0 msgr2=0x7fc138003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:36.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14ffff640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1380039b0 0x7fc138003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc14ffff640 1 -- 172.21.15.50:0/2048326331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc138003520 con 0x7fc138004800 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.458+0000 7fc15560d640 1 -- 172.21.15.50:0/2048326331 reap_dead start 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.459+0000 7fc14ffff640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc138004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc14400b9e0 tx=0x7fc138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ba20c29294bb73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.459+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144036020 con 0x7fc138004800 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.459+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc144032630 con 0x7fc138004800 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.459+0000 7fc14f7fe640 1 -- 172.21.15.50:0/2048326331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14400b5e0 con 0x7fc138004800 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.460+0000 7fc157070640 1 -- 172.21.15.50:0/2048326331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 msgr2=0x7fc138004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.460+0000 7fc157070640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc138004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc14400b9e0 tx=0x7fc138005830 comp rx=0 tx=0).stop 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.460+0000 7fc157070640 1 -- 172.21.15.50:0/2048326331 shutdown_connections 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.460+0000 7fc157070640 1 --2- 172.21.15.50:0/2048326331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc138004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.460+0000 7fc157070640 1 -- 172.21.15.50:0/2048326331 >> 172.21.15.50:0/2048326331 conn(0x7fc1500b3c20 msgr2=0x7fc1500b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.460+0000 7fc157070640 1 -- 172.21.15.50:0/2048326331 shutdown_connections 2024-04-02T16:21:36.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.460+0000 7fc157070640 1 -- 172.21.15.50:0/2048326331 wait complete. 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.461+0000 7fc157070640 1 Processor -- start 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.461+0000 7fc157070640 1 -- start start 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc157070640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc15006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc157070640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500b0130 0x7fc15006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc157070640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15006cb70 0x7fc150150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc157070640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500d0f20 con 0x7fc138004800 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc157070640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc15006fae0 con 0x7fc1500b0130 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc157070640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc1500ccc20 con 0x7fc15006cb70 2024-04-02T16:21:36.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc154e0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc15006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc154e0c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc15006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52098/0 (socket says 172.21.15.50:52098) 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc154e0c640 1 -- 172.21.15.50:0/62297021 learned_addr learned my addr 172.21.15.50:0/62297021 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc15560d640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15006cb70 0x7fc150150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.462+0000 7fc14ffff640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500b0130 0x7fc15006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc154e0c640 1 -- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15006cb70 msgr2=0x7fc150150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc154e0c640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15006cb70 0x7fc150150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc154e0c640 1 -- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500b0130 msgr2=0x7fc15006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc154e0c640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500b0130 0x7fc15006c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc154e0c640 1 -- 172.21.15.50:0/62297021 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15006e5f0 con 0x7fc138004800 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc154e0c640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc15006e0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc148001c40 tx=0x7fc14800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc14d7fa640 1 -- 172.21.15.50:0/62297021 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148019070 con 0x7fc138004800 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc14d7fa640 1 -- 172.21.15.50:0/62297021 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc148004d60 con 0x7fc138004800 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc14d7fa640 1 -- 172.21.15.50:0/62297021 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148007390 con 0x7fc138004800 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15006d010 con 0x7fc138004800 2024-04-02T16:21:36.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15006d5d0 con 0x7fc138004800 2024-04-02T16:21:36.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc14d7fa640 1 -- 172.21.15.50:0/62297021 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc14801c070 con 0x7fc138004800 2024-04-02T16:21:36.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.463+0000 7fc14d7fa640 1 -- 172.21.15.50:0/62297021 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc14801ebc0 con 0x7fc138004800 2024-04-02T16:21:36.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.464+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc150075760 con 0x7fc138004800 2024-04-02T16:21:36.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.467+0000 7fc14d7fa640 1 -- 172.21.15.50:0/62297021 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc148024020 con 0x7fc138004800 2024-04-02T16:21:36.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.534+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc1500b80b0 con 0x7fc138004800 2024-04-02T16:21:36.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.535+0000 7fc14d7fa640 1 -- 172.21.15.50:0/62297021 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fc1500ce110 con 0x7fc138004800 2024-04-02T16:21:36.536 INFO:teuthology.orchestra.run.smithi050.stderr:adjusted tunables profile to default 2024-04-02T16:21:36.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 msgr2=0x7fc15006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc15006e0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc148001c40 tx=0x7fc14800b4a0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 shutdown_connections 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15006cb70 0x7fc150150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500b0130 0x7fc15006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 --2- 172.21.15.50:0/62297021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc138004800 0x7fc15006e0b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 >> 172.21.15.50:0/62297021 conn(0x7fc1500b3c20 msgr2=0x7fc1500b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 shutdown_connections 2024-04-02T16:21:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:36.536+0000 7fc157070640 1 -- 172.21.15.50:0/62297021 wait complete. 2024-04-02T16:21:36.548 INFO:tasks.ceph:check_enable_crimson: False 2024-04-02T16:21:36.548 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-02T16:21:36.548 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:36.548 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-02T16:21:36.575 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:36.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-02T16:21:36.643 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:36.643 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-02T16:21:36.712 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:21:36.712 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-02T16:21:36.779 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:36.779 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-02T16:21:36.806 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:36.806 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-02T16:21:36.874 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:36.874 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-02T16:21:36.923 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:36.923+0000 7f888a02b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T16:21:36.940 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:21:36.941 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-02T16:21:36.979 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:36.978+0000 7f888a02b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T16:21:36.982 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:36.980+0000 7f0e82db9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T16:21:37.006 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 05cd5072-d309-4ce6-9b2b-dbbd6dab447e 0 2024-04-02T16:21:37.038 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.037+0000 7f0e82db9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T16:21:37.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.145+0000 7f402683c640 1 Processor -- start 2024-04-02T16:21:37.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.145+0000 7f402683c640 1 -- start start 2024-04-02T16:21:37.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f402683c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f402006bc30 0x7f40200b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f402683c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4020070590 0x7f40200b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f402683c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40200b5db0 con 0x7f402015e390 2024-04-02T16:21:37.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f402683c640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f40200b5ad0 con 0x7f402006bc30 2024-04-02T16:21:37.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f402683c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f40200b41c0 con 0x7f4020070590 2024-04-02T16:21:37.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f4024dd9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4020070590 0x7f40200b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f4024dd9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4020070590 0x7f40200b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59936/0 (socket says 172.21.15.188:59936) 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f4024dd9640 1 -- 172.21.15.188:0/241044371 learned_addr learned my addr 172.21.15.188:0/241044371 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f401f7fe640 1 --2- 172.21.15.188:0/241044371 >> v2:172.21.15.50:3300/0 conn(0x7f402006bc30 0x7f40200b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f4024dd9640 1 -- 172.21.15.188:0/241044371 >> v2:172.21.15.50:3300/0 conn(0x7f402006bc30 msgr2=0x7f40200b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.146+0000 7f4024dd9640 1 --2- 172.21.15.188:0/241044371 >> v2:172.21.15.50:3300/0 conn(0x7f402006bc30 0x7f40200b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.147+0000 7f4024dd9640 1 -- 172.21.15.188:0/241044371 >> v1:172.21.15.50:6789/0 conn(0x7f402015e390 legacy=0x7f402006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.147+0000 7f4024dd9640 1 -- 172.21.15.188:0/241044371 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40200b44a0 con 0x7f4020070590 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.147+0000 7f4024dd9640 1 --2- 172.21.15.188:0/241044371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4020070590 0x7f40200b51e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40200b64c0 tx=0x7f401802bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96243cfef1b28443 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.147+0000 7f401effd640 1 -- 172.21.15.188:0/241044371 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401802cb20 con 0x7f4020070590 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.147+0000 7f401effd640 1 -- 172.21.15.188:0/241044371 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401802ccc0 con 0x7f4020070590 2024-04-02T16:21:37.149 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.147+0000 7f401effd640 1 -- 172.21.15.188:0/241044371 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018034610 con 0x7f4020070590 2024-04-02T16:21:37.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 -- 172.21.15.188:0/241044371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4020070590 msgr2=0x7f40200b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 --2- 172.21.15.188:0/241044371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4020070590 0x7f40200b51e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40200b64c0 tx=0x7f401802bf80 comp rx=0 tx=0).stop 2024-04-02T16:21:37.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 -- 172.21.15.188:0/241044371 shutdown_connections 2024-04-02T16:21:37.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 --2- 172.21.15.188:0/241044371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4020070590 0x7f40200b51e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 --2- 172.21.15.188:0/241044371 >> v2:172.21.15.50:3300/0 conn(0x7f402006bc30 0x7f40200b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 -- 172.21.15.188:0/241044371 >> 172.21.15.188:0/241044371 conn(0x7f402006e680 msgr2=0x7f402006ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 -- 172.21.15.188:0/241044371 shutdown_connections 2024-04-02T16:21:37.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.148+0000 7f402683c640 1 -- 172.21.15.188:0/241044371 wait complete. 2024-04-02T16:21:37.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.149+0000 7f402683c640 1 Processor -- start 2024-04-02T16:21:37.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.149+0000 7f402683c640 1 -- start start 2024-04-02T16:21:37.153 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f402683c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402006bc30 0x7f40201542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.153 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f402683c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4020070590 0x7f40201f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.153 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f402683c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402015e390 0x7f40201f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.153 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f402683c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40200b5720 con 0x7f4020070590 2024-04-02T16:21:37.153 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f402683c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40200ae5d0 con 0x7f402015e390 2024-04-02T16:21:37.153 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f402683c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40201f4e70 con 0x7f402006bc30 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f4024dd9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402015e390 0x7f40201f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f4024dd9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402015e390 0x7f40201f13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59940/0 (socket says 172.21.15.188:59940) 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f401ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402006bc30 0x7f40201542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f4024dd9640 1 -- 172.21.15.188:0/2317397263 learned_addr learned my addr 172.21.15.188:0/2317397263 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.150+0000 7f401f7fe640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4020070590 0x7f40201f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f4024dd9640 1 -- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402006bc30 msgr2=0x7f40201542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f4024dd9640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402006bc30 0x7f40201542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f4024dd9640 1 -- 172.21.15.188:0/2317397263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4020070590 msgr2=0x7f40201f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f4024dd9640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4020070590 0x7f40201f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f4024dd9640 1 -- 172.21.15.188:0/2317397263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4020154880 con 0x7f402015e390 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f401ffff640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402006bc30 0x7f40201542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f401f7fe640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4020070590 0x7f40201f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f4024dd9640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402015e390 0x7f40201f13d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4018036040 tx=0x7f4018004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f401cff9640 1 -- 172.21.15.188:0/2317397263 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018043020 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f401cff9640 1 -- 172.21.15.188:0/2317397263 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401803e450 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4020154b60 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f401cff9640 1 -- 172.21.15.188:0/2317397263 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018009510 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4020155120 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f401cff9640 1 -- 172.21.15.188:0/2317397263 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4018046020 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.151+0000 7f401cff9640 1 -- 172.21.15.188:0/2317397263 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4018048400 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.152+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40200b6360 con 0x7f402015e390 2024-04-02T16:21:37.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.154+0000 7f401cff9640 1 -- 172.21.15.188:0/2317397263 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f40200b6360 con 0x7f402015e390 2024-04-02T16:21:37.225 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.224+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "05cd5072-d309-4ce6-9b2b-dbbd6dab447e", "id": 0} v 0) -- 0x7f40200756f0 con 0x7f402015e390 2024-04-02T16:21:37.228 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.227+0000 7f401cff9640 1 -- 172.21.15.188:0/2317397263 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "05cd5072-d309-4ce6-9b2b-dbbd6dab447e", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f401804d030 con 0x7f402015e390 2024-04-02T16:21:37.228 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-04-02T16:21:37.229 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402015e390 msgr2=0x7f40201f13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.229 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402015e390 0x7f40201f13d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4018036040 tx=0x7f4018004a40 comp rx=0 tx=0).stop 2024-04-02T16:21:37.229 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 shutdown_connections 2024-04-02T16:21:37.229 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402015e390 0x7f40201f13d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.229 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4020070590 0x7f40201f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.229 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 --2- 172.21.15.188:0/2317397263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402006bc30 0x7f40201542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.230 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 >> 172.21.15.188:0/2317397263 conn(0x7f402006e680 msgr2=0x7f402006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.230 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 shutdown_connections 2024-04-02T16:21:37.230 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.228+0000 7f402683c640 1 -- 172.21.15.188:0/2317397263 wait complete. 2024-04-02T16:21:37.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new dd26535d-1ca6-4547-974f-43bec7782174 1 2024-04-02T16:21:37.340 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.339+0000 7fd0f39bc640 1 Processor -- start 2024-04-02T16:21:37.340 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.339+0000 7fd0f39bc640 1 -- start start 2024-04-02T16:21:37.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f39bc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd0ec15b890 0x7fd0ec071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f39bc640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ec15bd20 con 0x7fd0ec0b5320 2024-04-02T16:21:37.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f39bc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ec072660 con 0x7fd0ec15a710 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f39bc640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fd0ec072380 con 0x7fd0ec15b890 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f1758640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd0ec15a710 0x7fd0ec0b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:58466/0 (socket says 172.21.15.188:58466) 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f1758640 1 -- 172.21.15.188:0/2125223929 learned_addr learned my addr 172.21.15.188:0/2125223929 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f0f57640 1 --2- 172.21.15.188:0/2125223929 >> v2:172.21.15.50:3300/0 conn(0x7fd0ec15b890 0x7fd0ec071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0e3fff640 1 -- 172.21.15.188:0/2125223929 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973358232 0 0) 0x7fd0ec072660 con 0x7fd0ec15a710 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0e3fff640 1 -- 172.21.15.188:0/2125223929 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0d8003520 con 0x7fd0ec15a710 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f0f57640 1 -- 172.21.15.188:0/2125223929 >> v1:172.21.15.188:6789/0 conn(0x7fd0ec15a710 legacy=0x7fd0ec0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f0f57640 1 -- 172.21.15.188:0/2125223929 >> v1:172.21.15.50:6789/0 conn(0x7fd0ec0b5320 legacy=0x7fd0ec0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.340+0000 7fd0f0f57640 1 -- 172.21.15.188:0/2125223929 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec0b3890 con 0x7fd0ec15b890 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.341+0000 7fd0f0f57640 1 --2- 172.21.15.188:0/2125223929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15b890 0x7fd0ec071c30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0097a0 tx=0x7fd0dc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da8a756006212fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.341+0000 7fd0e3fff640 1 -- 172.21.15.188:0/2125223929 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc034070 con 0x7fd0ec15b890 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.341+0000 7fd0e3fff640 1 -- 172.21.15.188:0/2125223929 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc030610 con 0x7fd0ec15b890 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.341+0000 7fd0e3fff640 1 -- 172.21.15.188:0/2125223929 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0309c0 con 0x7fd0ec15b890 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.342+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2125223929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15b890 msgr2=0x7fd0ec071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.342+0000 7fd0f39bc640 1 --2- 172.21.15.188:0/2125223929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15b890 0x7fd0ec071c30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0097a0 tx=0x7fd0dc02b960 comp rx=0 tx=0).stop 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.342+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2125223929 shutdown_connections 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.342+0000 7fd0f39bc640 1 --2- 172.21.15.188:0/2125223929 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15b890 0x7fd0ec071c30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.342+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2125223929 >> 172.21.15.188:0/2125223929 conn(0x7fd0ec06b520 msgr2=0x7fd0ec06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.342+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2125223929 shutdown_connections 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.342+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2125223929 wait complete. 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.343+0000 7fd0f39bc640 1 Processor -- start 2024-04-02T16:21:37.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.343+0000 7fd0f39bc640 1 -- start start 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f39bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0ec0b5320 0x7fd0ec0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f39bc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15a710 0x7fd0ec150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f39bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0ec15b890 0x7fd0ec0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f39bc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec15aaf0 con 0x7fd0ec15a710 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f39bc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec0b3e50 con 0x7fd0ec0b5320 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f39bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec15ac70 con 0x7fd0ec15b890 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1758640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0ec0b5320 0x7fd0ec0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1758640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0ec0b5320 0x7fd0ec0cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59946/0 (socket says 172.21.15.188:59946) 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1758640 1 -- 172.21.15.188:0/2308561011 learned_addr learned my addr 172.21.15.188:0/2308561011 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1f59640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0ec15b890 0x7fd0ec0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f0f57640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15a710 0x7fd0ec150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1f59640 1 -- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0ec0b5320 msgr2=0x7fd0ec0cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1f59640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0ec0b5320 0x7fd0ec0cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1f59640 1 -- 172.21.15.188:0/2308561011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15a710 msgr2=0x7fd0ec150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1f59640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15a710 0x7fd0ec150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1f59640 1 -- 172.21.15.188:0/2308561011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec150660 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1758640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0ec0b5320 0x7fd0ec0cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0f1f59640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0ec15b890 0x7fd0ec0ce7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c690 tx=0x7fd0e400cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0e1ffb640 1 -- 172.21.15.188:0/2308561011 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4003b70 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.344+0000 7fd0e1ffb640 1 -- 172.21.15.188:0/2308561011 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e4003d10 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.345+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec152cb0 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.345+0000 7fd0e1ffb640 1 -- 172.21.15.188:0/2308561011 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4005310 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.345+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec153270 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.345+0000 7fd0e1ffb640 1 -- 172.21.15.188:0/2308561011 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd0e40054b0 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.345+0000 7fd0e1ffb640 1 -- 172.21.15.188:0/2308561011 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd0e4012540 con 0x7fd0ec15b890 2024-04-02T16:21:37.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.346+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ec0756f0 con 0x7fd0ec15b890 2024-04-02T16:21:37.349 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.348+0000 7fd0e1ffb640 1 -- 172.21.15.188:0/2308561011 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd0ec15bd20 con 0x7fd0ec15b890 2024-04-02T16:21:37.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.420+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd26535d-1ca6-4547-974f-43bec7782174", "id": 1} v 0) -- 0x7fd0ec071e10 con 0x7fd0ec15b890 2024-04-02T16:21:37.424 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.423+0000 7fd0e1ffb640 1 -- 172.21.15.188:0/2308561011 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd26535d-1ca6-4547-974f-43bec7782174", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd0e400a640 con 0x7fd0ec15b890 2024-04-02T16:21:37.424 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0ec15b890 msgr2=0x7fd0ec0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0ec15b890 0x7fd0ec0ce7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400c690 tx=0x7fd0e400cb50 comp rx=0 tx=0).stop 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 shutdown_connections 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0ec15b890 0x7fd0ec0ce7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec15a710 0x7fd0ec150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 --2- 172.21.15.188:0/2308561011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0ec0b5320 0x7fd0ec0cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 >> 172.21.15.188:0/2308561011 conn(0x7fd0ec06b520 msgr2=0x7fd0ec152280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 shutdown_connections 2024-04-02T16:21:37.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.424+0000 7fd0f39bc640 1 -- 172.21.15.188:0/2308561011 wait complete. 2024-04-02T16:21:37.436 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 54c49032-e743-4090-8c7b-a09f6bd14a69 2 2024-04-02T16:21:37.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.536+0000 7f3590893640 1 Processor -- start 2024-04-02T16:21:37.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.536+0000 7f3590893640 1 -- start start 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3590893640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f358c0d2bd0 0x7f358c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3590893640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c0d1060 0x7f358c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3590893640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f358c15e9f0 con 0x7f358c15e610 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3590893640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f358c06c9e0 con 0x7f358c0d2bd0 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3590893640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f358c06ccc0 con 0x7f358c0d1060 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3583fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f358c0d2bd0 0x7f358c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f358ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c0d1060 0x7f358c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3583fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f358c0d2bd0 0x7f358c0d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59954/0 (socket says 172.21.15.188:59954) 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f358ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c0d1060 0x7f358c09aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:42914/0 (socket says 172.21.15.188:42914) 2024-04-02T16:21:37.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.537+0000 7f3583fff640 1 -- 172.21.15.188:0/523764225 learned_addr learned my addr 172.21.15.188:0/523764225 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.538+0000 7f358ad9d640 1 -- 172.21.15.188:0/523764225 >> v2:172.21.15.188:3300/0 conn(0x7f358c0d2bd0 msgr2=0x7f358c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:37.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.538+0000 7f358ad9d640 1 --2- 172.21.15.188:0/523764225 >> v2:172.21.15.188:3300/0 conn(0x7f358c0d2bd0 0x7f358c0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.538+0000 7f358ad9d640 1 -- 172.21.15.188:0/523764225 >> v1:172.21.15.50:6789/0 conn(0x7f358c15e610 legacy=0x7f358c0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.538+0000 7f358ad9d640 1 -- 172.21.15.188:0/523764225 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c0d0600 con 0x7f358c0d1060 2024-04-02T16:21:37.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.538+0000 7f3583fff640 1 --2- 172.21.15.188:0/523764225 >> v2:172.21.15.188:3300/0 conn(0x7f358c0d2bd0 0x7f358c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.538+0000 7f358ad9d640 1 --2- 172.21.15.188:0/523764225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c0d1060 0x7f358c09aa40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3584009d90 tx=0x7f358402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b0f58394caeffd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3589d9b640 1 -- 172.21.15.188:0/523764225 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584039070 con 0x7f358c0d1060 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3589d9b640 1 -- 172.21.15.188:0/523764225 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358402cce0 con 0x7f358c0d1060 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3589d9b640 1 -- 172.21.15.188:0/523764225 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584034660 con 0x7f358c0d1060 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 -- 172.21.15.188:0/523764225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c0d1060 msgr2=0x7f358c09aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 --2- 172.21.15.188:0/523764225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c0d1060 0x7f358c09aa40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3584009d90 tx=0x7f358402bf30 comp rx=0 tx=0).stop 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 -- 172.21.15.188:0/523764225 shutdown_connections 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 --2- 172.21.15.188:0/523764225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c0d1060 0x7f358c09aa40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.541 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 --2- 172.21.15.188:0/523764225 >> v2:172.21.15.188:3300/0 conn(0x7f358c0d2bd0 0x7f358c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 -- 172.21.15.188:0/523764225 >> 172.21.15.188:0/523764225 conn(0x7f358c0d20b0 msgr2=0x7f358c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 -- 172.21.15.188:0/523764225 shutdown_connections 2024-04-02T16:21:37.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.539+0000 7f3590893640 1 -- 172.21.15.188:0/523764225 wait complete. 2024-04-02T16:21:37.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 Processor -- start 2024-04-02T16:21:37.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 -- start start 2024-04-02T16:21:37.543 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f888a02b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f358c0d1060 0x7f358c09fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c0d2bd0 0x7f358c09f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c15e610 0x7f358c09d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f358c0d1ea0 con 0x7f358c0d1060 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f358c0d1d20 con 0x7f358c0d2bd0 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.541+0000 7f3590893640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f358c0d14f0 con 0x7f358c15e610 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c0d2bd0 0x7f358c09f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f358ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c15e610 0x7f358c09d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c0d2bd0 0x7f358c09f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59962/0 (socket says 172.21.15.188:59962) 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 -- 172.21.15.188:0/3035834680 learned_addr learned my addr 172.21.15.188:0/3035834680 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f358a59c640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f358c0d1060 0x7f358c09fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 -- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c15e610 msgr2=0x7f358c09d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c15e610 0x7f358c09d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 -- 172.21.15.188:0/3035834680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f358c0d1060 msgr2=0x7f358c09fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f358c0d1060 0x7f358c09fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 -- 172.21.15.188:0/3035834680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c09f740 con 0x7f358c0d2bd0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f358ad9d640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c15e610 0x7f358c09d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3583fff640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c0d2bd0 0x7f358c09f200 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f357400ea00 tx=0x7f357400eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3582ffd640 1 -- 172.21.15.188:0/3035834680 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574017070 con 0x7f358c0d2bd0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3582ffd640 1 -- 172.21.15.188:0/3035834680 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f357400ca60 con 0x7f358c0d2bd0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c09de90 con 0x7f358c0d2bd0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3582ffd640 1 -- 172.21.15.188:0/3035834680 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357400ce10 con 0x7f358c0d2bd0 2024-04-02T16:21:37.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c09e440 con 0x7f358c0d2bd0 2024-04-02T16:21:37.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3582ffd640 1 -- 172.21.15.188:0/3035834680 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f357401a070 con 0x7f358c0d2bd0 2024-04-02T16:21:37.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.542+0000 7f3582ffd640 1 -- 172.21.15.188:0/3035834680 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3574013470 con 0x7f358c0d2bd0 2024-04-02T16:21:37.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.543+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c0756f0 con 0x7f358c0d2bd0 2024-04-02T16:21:37.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.545+0000 7f3582ffd640 1 -- 172.21.15.188:0/3035834680 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3574006020 con 0x7f358c0d2bd0 2024-04-02T16:21:37.593 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.592+0000 7f0e82db9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T16:21:37.599 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.598+0000 7f888a02b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T16:21:37.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.618+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "54c49032-e743-4090-8c7b-a09f6bd14a69", "id": 2} v 0) -- 0x7f358c0d2810 con 0x7f358c0d2bd0 2024-04-02T16:21:37.622 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.621+0000 7f3582ffd640 1 -- 172.21.15.188:0/3035834680 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "54c49032-e743-4090-8c7b-a09f6bd14a69", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3574006570 con 0x7f358c0d2bd0 2024-04-02T16:21:37.622 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-04-02T16:21:37.623 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c0d2bd0 msgr2=0x7f358c09f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.623 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c0d2bd0 0x7f358c09f200 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f357400ea00 tx=0x7f357400eec0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.623 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 shutdown_connections 2024-04-02T16:21:37.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c15e610 0x7f358c09d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c0d2bd0 0x7f358c09f200 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 --2- 172.21.15.188:0/3035834680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f358c0d1060 0x7f358c09fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 >> 172.21.15.188:0/3035834680 conn(0x7f358c0d20b0 msgr2=0x7f358c0c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 shutdown_connections 2024-04-02T16:21:37.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.622+0000 7f3590893640 1 -- 172.21.15.188:0/3035834680 wait complete. 2024-04-02T16:21:37.633 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new ad773d48-a65b-4b99-8d48-8b8bdf1f4853 3 2024-04-02T16:21:37.646 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.646+0000 7f888a02b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T16:21:37.650 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.649+0000 7f0e82db9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T16:21:37.697 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.696+0000 7f0e82db9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T16:21:37.704 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.703+0000 7f888a02b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T16:21:37.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.733+0000 7ff588e47640 1 Processor -- start 2024-04-02T16:21:37.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff588e47640 1 -- start start 2024-04-02T16:21:37.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff588e47640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff58415e610 0x7ff5840cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff588e47640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5840d1230 0x7ff58409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff588e47640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5840ccc40 con 0x7ff5840d2bd0 2024-04-02T16:21:37.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff588e47640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ff58415e9f0 con 0x7ff58415e610 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff588e47640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff5840d16c0 con 0x7ff5840d1230 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff582d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5840d1230 0x7ff58409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff582d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5840d1230 0x7ff58409aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59978/0 (socket says 172.21.15.188:59978) 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff582d9d640 1 -- 172.21.15.188:0/2402448372 learned_addr learned my addr 172.21.15.188:0/2402448372 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.734+0000 7ff57bfff640 1 --1- 172.21.15.188:0/2402448372 >> v1:172.21.15.188:6789/0 conn(0x7ff5840d2bd0 0x7ff5840d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:58482/0 (socket says 172.21.15.188:58482) 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.735+0000 7ff58259c640 1 --2- 172.21.15.188:0/2402448372 >> v2:172.21.15.50:3300/0 conn(0x7ff58415e610 0x7ff5840cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.735+0000 7ff582d9d640 1 -- 172.21.15.188:0/2402448372 >> v2:172.21.15.50:3300/0 conn(0x7ff58415e610 msgr2=0x7ff5840cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.735+0000 7ff582d9d640 1 --2- 172.21.15.188:0/2402448372 >> v2:172.21.15.50:3300/0 conn(0x7ff58415e610 0x7ff5840cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.735+0000 7ff582d9d640 1 -- 172.21.15.188:0/2402448372 >> v1:172.21.15.188:6789/0 conn(0x7ff5840d2bd0 legacy=0x7ff5840d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.735+0000 7ff582d9d640 1 -- 172.21.15.188:0/2402448372 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58406c9e0 con 0x7ff5840d1230 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.735+0000 7ff582d9d640 1 --2- 172.21.15.188:0/2402448372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d1230 0x7ff58409aa40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff58409bb00 tx=0x7ff57c02c010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f63b9f47b2ef61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff581d9b640 1 -- 172.21.15.188:0/2402448372 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c02cbb0 con 0x7ff5840d1230 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff581d9b640 1 -- 172.21.15.188:0/2402448372 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c02cd50 con 0x7ff5840d1230 2024-04-02T16:21:37.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff581d9b640 1 -- 172.21.15.188:0/2402448372 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c034610 con 0x7ff5840d1230 2024-04-02T16:21:37.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 -- 172.21.15.188:0/2402448372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d1230 msgr2=0x7ff58409aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 --2- 172.21.15.188:0/2402448372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d1230 0x7ff58409aa40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff58409bb00 tx=0x7ff57c02c010 comp rx=0 tx=0).stop 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 -- 172.21.15.188:0/2402448372 shutdown_connections 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 --2- 172.21.15.188:0/2402448372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d1230 0x7ff58409aa40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 --2- 172.21.15.188:0/2402448372 >> v2:172.21.15.50:3300/0 conn(0x7ff58415e610 0x7ff5840cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 -- 172.21.15.188:0/2402448372 >> 172.21.15.188:0/2402448372 conn(0x7ff5840d20b0 msgr2=0x7ff5840d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 -- 172.21.15.188:0/2402448372 shutdown_connections 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.736+0000 7ff588e47640 1 -- 172.21.15.188:0/2402448372 wait complete. 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 Processor -- start 2024-04-02T16:21:37.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 -- start start 2024-04-02T16:21:37.741 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5840d1230 0x7ff5840b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.741 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d2bd0 0x7ff5840c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.741 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58415e610 0x7ff5840c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.741 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff5840d0240 con 0x7ff5840d1230 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5840d0600 con 0x7ff5840d2bd0 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.738+0000 7ff588e47640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff5840d03c0 con 0x7ff58415e610 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d2bd0 0x7ff5840c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58415e610 0x7ff5840c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58415e610 0x7ff5840c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:42934/0 (socket says 172.21.15.188:42934) 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d2bd0 0x7ff5840c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:59988/0 (socket says 172.21.15.188:59988) 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 -- 172.21.15.188:0/1344195533 learned_addr learned my addr 172.21.15.188:0/1344195533 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff58259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5840d1230 0x7ff5840b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 -- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d2bd0 msgr2=0x7ff5840c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d2bd0 0x7ff5840c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 -- 172.21.15.188:0/1344195533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5840d1230 msgr2=0x7ff5840b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5840d1230 0x7ff5840b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 -- 172.21.15.188:0/1344195533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff584161620 con 0x7ff58415e610 2024-04-02T16:21:37.742 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff58259c640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5840d1230 0x7ff5840b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57bfff640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d2bd0 0x7ff5840c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff582d9d640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58415e610 0x7ff5840c9b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff57c036040 tx=0x7ff57c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57affd640 1 -- 172.21.15.188:0/1344195533 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c043020 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57affd640 1 -- 172.21.15.188:0/1344195533 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c03e450 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff584161900 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57affd640 1 -- 172.21.15.188:0/1344195533 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c009510 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5840b6b00 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57affd640 1 -- 172.21.15.188:0/1344195533 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff57c046020 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.739+0000 7ff57affd640 1 -- 172.21.15.188:0/1344195533 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff57c048890 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.740+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5840756f0 con 0x7ff58415e610 2024-04-02T16:21:37.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.742+0000 7ff57affd640 1 -- 172.21.15.188:0/1344195533 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff57c04d030 con 0x7ff58415e610 2024-04-02T16:21:37.749 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.749+0000 7f888a02b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T16:21:37.753 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.752+0000 7f0e82db9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T16:21:37.797 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.796+0000 7f0e82db9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T16:21:37.815 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.815+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ad773d48-a65b-4b99-8d48-8b8bdf1f4853", "id": 3} v 0) -- 0x7ff5840b10a0 con 0x7ff58415e610 2024-04-02T16:21:37.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.818+0000 7ff57affd640 1 -- 172.21.15.188:0/1344195533 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad773d48-a65b-4b99-8d48-8b8bdf1f4853", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff57c02cbc0 con 0x7ff58415e610 2024-04-02T16:21:37.819 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58415e610 msgr2=0x7ff5840c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58415e610 0x7ff5840c9b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff57c036040 tx=0x7ff57c004a40 comp rx=0 tx=0).stop 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 shutdown_connections 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58415e610 0x7ff5840c9b90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5840d2bd0 0x7ff5840c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 --2- 172.21.15.188:0/1344195533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5840d1230 0x7ff5840b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 >> 172.21.15.188:0/1344195533 conn(0x7ff5840d20b0 msgr2=0x7ff5840ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 shutdown_connections 2024-04-02T16:21:37.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.819+0000 7ff588e47640 1 -- 172.21.15.188:0/1344195533 wait complete. 2024-04-02T16:21:37.830 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 7d37ca27-5a6f-4c8e-a029-3076f76e0166 4 2024-04-02T16:21:37.871 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.870+0000 7f888a02b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T16:21:37.920 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.919+0000 7f0e82db9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T16:21:37.945 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.944+0000 7f888a02b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T16:21:37.971 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.970+0000 7f594bed5640 1 Processor -- start 2024-04-02T16:21:37.972 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.971+0000 7f594bed5640 1 -- start start 2024-04-02T16:21:37.973 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.971+0000 7f594bed5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f59440d1230 0x7f5944169430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.973 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.971+0000 7f594bed5640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594415e9f0 con 0x7f594415e610 2024-04-02T16:21:37.973 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.971+0000 7f594bed5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59440d16c0 con 0x7f59440d2bd0 2024-04-02T16:21:37.973 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.971+0000 7f594bed5640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f594406c9e0 con 0x7f59440d1230 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f5949470640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f59440d2bd0 0x7f59440d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:58490/0 (socket says 172.21.15.188:58490) 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f5949470640 1 -- 172.21.15.188:0/2505962854 learned_addr learned my addr 172.21.15.188:0/2505962854 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f594a472640 1 --2- 172.21.15.188:0/2505962854 >> v2:172.21.15.50:3300/0 conn(0x7f59440d1230 0x7f5944169430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f5948c6f640 1 -- 172.21.15.188:0/2505962854 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579053344 0 0) 0x7f59440d16c0 con 0x7f59440d2bd0 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f5948c6f640 1 -- 172.21.15.188:0/2505962854 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5930003520 con 0x7f59440d2bd0 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f5948c6f640 1 -- 172.21.15.188:0/2505962854 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992688455 0 0) 0x7f594415e9f0 con 0x7f594415e610 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f5948c6f640 1 -- 172.21.15.188:0/2505962854 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59440d16c0 con 0x7f594415e610 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f594a472640 1 -- 172.21.15.188:0/2505962854 >> v1:172.21.15.188:6789/0 conn(0x7f59440d2bd0 legacy=0x7f59440d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f594a472640 1 -- 172.21.15.188:0/2505962854 >> v1:172.21.15.50:6789/0 conn(0x7f594415e610 legacy=0x7f59440cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.974 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.972+0000 7f594a472640 1 -- 172.21.15.188:0/2505962854 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594406ccc0 con 0x7f59440d1230 2024-04-02T16:21:37.975 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.973+0000 7f594a472640 1 --2- 172.21.15.188:0/2505962854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d1230 0x7f5944169430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f593c009d90 tx=0x7f593c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40492aaa05fc9bae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.973+0000 7f5948c6f640 1 -- 172.21.15.188:0/2505962854 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c039070 con 0x7f59440d1230 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f5948c6f640 1 -- 172.21.15.188:0/2505962854 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593c02cd80 con 0x7f59440d1230 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f5948c6f640 1 -- 172.21.15.188:0/2505962854 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c034660 con 0x7f59440d1230 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f594bed5640 1 -- 172.21.15.188:0/2505962854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d1230 msgr2=0x7f5944169430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f594bed5640 1 --2- 172.21.15.188:0/2505962854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d1230 0x7f5944169430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f593c009d90 tx=0x7f593c02bfd0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f594bed5640 1 -- 172.21.15.188:0/2505962854 shutdown_connections 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f594bed5640 1 --2- 172.21.15.188:0/2505962854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d1230 0x7f5944169430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f594bed5640 1 -- 172.21.15.188:0/2505962854 >> 172.21.15.188:0/2505962854 conn(0x7f59440d20b0 msgr2=0x7f59440d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f594bed5640 1 -- 172.21.15.188:0/2505962854 shutdown_connections 2024-04-02T16:21:37.976 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.974+0000 7f594bed5640 1 -- 172.21.15.188:0/2505962854 wait complete. 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.975+0000 7f594bed5640 1 Processor -- start 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.975+0000 7f594bed5640 1 -- start start 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f594bed5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59440d1230 0x7f594414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f594bed5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d2bd0 0x7f594409e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f594bed5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594415e610 0x7f59440c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f594bed5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f59440d03c0 con 0x7f59440d2bd0 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f594bed5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59440d0240 con 0x7f594415e610 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f594bed5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f59440d1d20 con 0x7f59440d1230 2024-04-02T16:21:37.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f5949c71640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59440d1230 0x7f594414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f594a472640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594415e610 0x7f59440c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f5949c71640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59440d1230 0x7f594414f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:42946/0 (socket says 172.21.15.188:42946) 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f5949c71640 1 -- 172.21.15.188:0/3566138059 learned_addr learned my addr 172.21.15.188:0/3566138059 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.976+0000 7f5949470640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d2bd0 0x7f594409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f5949c71640 1 -- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594415e610 msgr2=0x7f59440c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f5949c71640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594415e610 0x7f59440c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f5949c71640 1 -- 172.21.15.188:0/3566138059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d2bd0 msgr2=0x7f594409e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f5949c71640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d2bd0 0x7f594409e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f5949c71640 1 -- 172.21.15.188:0/3566138059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594409f310 con 0x7f59440d1230 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f594a472640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594415e610 0x7f59440c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f5949470640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d2bd0 0x7f594409e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f5949c71640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59440d1230 0x7f594414f590 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f594000c460 tx=0x7f594000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f593a7fc640 1 -- 172.21.15.188:0/3566138059 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940017070 con 0x7f59440d1230 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f593a7fc640 1 -- 172.21.15.188:0/3566138059 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59400052f0 con 0x7f59440d1230 2024-04-02T16:21:37.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594409f5f0 con 0x7f59440d1230 2024-04-02T16:21:37.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f593a7fc640 1 -- 172.21.15.188:0/3566138059 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940003830 con 0x7f59440d1230 2024-04-02T16:21:37.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f594409c280 con 0x7f59440d1230 2024-04-02T16:21:37.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f593a7fc640 1 -- 172.21.15.188:0/3566138059 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f594001a070 con 0x7f59440d1230 2024-04-02T16:21:37.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.977+0000 7f593a7fc640 1 -- 172.21.15.188:0/3566138059 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5940011c20 con 0x7f59440d1230 2024-04-02T16:21:37.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.978+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594409fb10 con 0x7f59440d1230 2024-04-02T16:21:37.982 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:37.982+0000 7f593a7fc640 1 -- 172.21.15.188:0/3566138059 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f594409fb10 con 0x7f59440d1230 2024-04-02T16:21:37.993 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:37.992+0000 7f0e82db9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T16:21:37.996 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:37.995+0000 7f888a02b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T16:21:38.041 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:38.040+0000 7f0e82db9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T16:21:38.050 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:38.049+0000 7f888a02b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T16:21:38.055 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.055+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7d37ca27-5a6f-4c8e-a029-3076f76e0166", "id": 4} v 0) -- 0x7f59440756f0 con 0x7f59440d1230 2024-04-02T16:21:38.059 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.058+0000 7f593a7fc640 1 -- 172.21.15.188:0/3566138059 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d37ca27-5a6f-4c8e-a029-3076f76e0166", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f5940021070 con 0x7f59440d1230 2024-04-02T16:21:38.059 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-04-02T16:21:38.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59440d1230 msgr2=0x7f594414f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59440d1230 0x7f594414f590 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f594000c460 tx=0x7f594000c920 comp rx=0 tx=0).stop 2024-04-02T16:21:38.060 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 shutdown_connections 2024-04-02T16:21:38.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594415e610 0x7f59440c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59440d2bd0 0x7f594409e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 --2- 172.21.15.188:0/3566138059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59440d1230 0x7f594414f590 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 >> 172.21.15.188:0/3566138059 conn(0x7f59440d20b0 msgr2=0x7f594415f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 shutdown_connections 2024-04-02T16:21:38.061 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.059+0000 7f594bed5640 1 -- 172.21.15.188:0/3566138059 wait complete. 2024-04-02T16:21:38.071 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new f4e15f5c-b81a-46b9-884d-b0fc57937947 5 2024-04-02T16:21:38.096 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:38.095+0000 7f0e82db9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T16:21:38.103 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:38.102+0000 7f888a02b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T16:21:38.145 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:38.144+0000 7f0e82db9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T16:21:38.172 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.171+0000 7f210141b640 1 Processor -- start 2024-04-02T16:21:38.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f210141b640 1 -- start start 2024-04-02T16:21:38.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f210141b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f20fc15e610 0x7f20fc0d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f210141b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20fc0d4c90 0x7f20fc0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f210141b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20fc15e9f0 con 0x7f20fc0d1180 2024-04-02T16:21:38.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f210141b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f20fc0d5120 con 0x7f20fc15e610 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f210141b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f20fc0d3210 con 0x7f20fc0d4c90 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f20fa7fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20fc0d4c90 0x7f20fc0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f20fa7fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20fc0d4c90 0x7f20fc0d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:60018/0 (socket says 172.21.15.188:60018) 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f20fa7fc640 1 -- 172.21.15.188:0/1776958240 learned_addr learned my addr 172.21.15.188:0/1776958240 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.172+0000 7f20fb7fe640 1 --1- 172.21.15.188:0/1776958240 >> v1:172.21.15.188:6789/0 conn(0x7f20fc0d1180 0x7f20fc0ced80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:58498/0 (socket says 172.21.15.188:58498) 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.173+0000 7f20faffd640 1 --2- 172.21.15.188:0/1776958240 >> v2:172.21.15.50:3300/0 conn(0x7f20fc15e610 0x7f20fc0d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.173+0000 7f20fa7fc640 1 -- 172.21.15.188:0/1776958240 >> v2:172.21.15.50:3300/0 conn(0x7f20fc15e610 msgr2=0x7f20fc0d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.173+0000 7f20fa7fc640 1 --2- 172.21.15.188:0/1776958240 >> v2:172.21.15.50:3300/0 conn(0x7f20fc15e610 0x7f20fc0d2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.173+0000 7f20fa7fc640 1 -- 172.21.15.188:0/1776958240 >> v1:172.21.15.188:6789/0 conn(0x7f20fc0d1180 legacy=0x7f20fc0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.173+0000 7f20fa7fc640 1 -- 172.21.15.188:0/1776958240 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc0d2f30 con 0x7f20fc0d4c90 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.173+0000 7f20f9ffb640 1 -- 172.21.15.188:0/1776958240 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2287691954 0 0) 0x7f20fc15e9f0 con 0x7f20fc0d1180 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.173+0000 7f20fa7fc640 1 --2- 172.21.15.188:0/1776958240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc0d4c90 0x7f20fc0d0c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f20fc08cb20 tx=0x7f20ec02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d59da9676d492c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f20f9ffb640 1 -- 172.21.15.188:0/1776958240 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec034070 con 0x7f20fc0d4c90 2024-04-02T16:21:38.175 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f20f9ffb640 1 -- 172.21.15.188:0/1776958240 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec030860 con 0x7f20fc0d4c90 2024-04-02T16:21:38.176 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f20f9ffb640 1 -- 172.21.15.188:0/1776958240 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec030c30 con 0x7f20fc0d4c90 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 -- 172.21.15.188:0/1776958240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc0d4c90 msgr2=0x7f20fc0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 --2- 172.21.15.188:0/1776958240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc0d4c90 0x7f20fc0d0c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f20fc08cb20 tx=0x7f20ec02b9a0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 -- 172.21.15.188:0/1776958240 shutdown_connections 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 --2- 172.21.15.188:0/1776958240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc0d4c90 0x7f20fc0d0c40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 --2- 172.21.15.188:0/1776958240 >> v2:172.21.15.50:3300/0 conn(0x7f20fc15e610 0x7f20fc0d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 -- 172.21.15.188:0/1776958240 >> 172.21.15.188:0/1776958240 conn(0x7f20fc09d550 msgr2=0x7f20fc0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 -- 172.21.15.188:0/1776958240 shutdown_connections 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.174+0000 7f210141b640 1 -- 172.21.15.188:0/1776958240 wait complete. 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.175+0000 7f210141b640 1 Processor -- start 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f210141b640 1 -- start start 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f210141b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20fc0d1180 0x7f20fc153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f210141b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0d4c90 0x7f20fc15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f210141b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc15e610 0x7f20fc0b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f210141b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc0b4310 con 0x7f20fc0d1180 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f210141b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc0b4030 con 0x7f20fc15e610 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f210141b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f20fc0b5840 con 0x7f20fc0d4c90 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f20fb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc15e610 0x7f20fc0b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f20fa7fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0d4c90 0x7f20fc15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f20fb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc15e610 0x7f20fc0b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:60026/0 (socket says 172.21.15.188:60026) 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f20fb7fe640 1 -- 172.21.15.188:0/2422875554 learned_addr learned my addr 172.21.15.188:0/2422875554 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.176+0000 7f20faffd640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20fc0d1180 0x7f20fc153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20fb7fe640 1 -- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0d4c90 msgr2=0x7f20fc15ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20fb7fe640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0d4c90 0x7f20fc15ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20fb7fe640 1 -- 172.21.15.188:0/2422875554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20fc0d1180 msgr2=0x7f20fc153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20fb7fe640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20fc0d1180 0x7f20fc153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20fb7fe640 1 -- 172.21.15.188:0/2422875554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc15ef30 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20faffd640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20fc0d1180 0x7f20fc153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20fa7fc640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0d4c90 0x7f20fc15ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20fb7fe640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc15e610 0x7f20fc0b8820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c7a0 tx=0x7f20f400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20e37fe640 1 -- 172.21.15.188:0/2422875554 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4018070 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20e37fe640 1 -- 172.21.15.188:0/2422875554 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f400a640 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc15f210 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20e37fe640 1 -- 172.21.15.188:0/2422875554 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f400aab0 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc15f7d0 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20e37fe640 1 -- 172.21.15.188:0/2422875554 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20f401b070 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.177+0000 7f20e37fe640 1 -- 172.21.15.188:0/2422875554 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f20f40078f0 con 0x7f20fc15e610 2024-04-02T16:21:38.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.178+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20fc0756f0 con 0x7f20fc15e610 2024-04-02T16:21:38.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.182+0000 7f20e37fe640 1 -- 172.21.15.188:0/2422875554 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f20f400acb0 con 0x7f20fc15e610 2024-04-02T16:21:38.255 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.254+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f4e15f5c-b81a-46b9-884d-b0fc57937947", "id": 5} v 0) -- 0x7f20fc08dfd0 con 0x7f20fc15e610 2024-04-02T16:21:38.258 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.258+0000 7f20e37fe640 1 -- 172.21.15.188:0/2422875554 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4e15f5c-b81a-46b9-884d-b0fc57937947", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f20f40070d0 con 0x7f20fc15e610 2024-04-02T16:21:38.259 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc15e610 msgr2=0x7f20fc0b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc15e610 0x7f20fc0b8820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c7a0 tx=0x7f20f400cc60 comp rx=0 tx=0).stop 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 shutdown_connections 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc15e610 0x7f20fc0b8820 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0d4c90 0x7f20fc15ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 --2- 172.21.15.188:0/2422875554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20fc0d1180 0x7f20fc153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 >> 172.21.15.188:0/2422875554 conn(0x7f20fc09d550 msgr2=0x7f20fc0cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 shutdown_connections 2024-04-02T16:21:38.260 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.259+0000 7f210141b640 1 -- 172.21.15.188:0/2422875554 wait complete. 2024-04-02T16:21:38.270 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 427efa9c-ad3f-4f2b-8205-f1640d4e55ef 6 2024-04-02T16:21:38.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.411+0000 7f112fe47640 1 Processor -- start 2024-04-02T16:21:38.412 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.411+0000 7f112fe47640 1 -- start start 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112fe47640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f112806bc30 0x7f11280b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112fe47640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11280b5db0 con 0x7f112815e390 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112fe47640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11280b5ad0 con 0x7f1128070590 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112fe47640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f11280b41c0 con 0x7f112806bc30 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112e3e4640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1128070590 0x7f11280b51e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:58500/0 (socket says 172.21.15.188:58500) 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112e3e4640 1 -- 172.21.15.188:0/2464898286 learned_addr learned my addr 172.21.15.188:0/2464898286 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112d3e2640 1 --2- 172.21.15.188:0/2464898286 >> v2:172.21.15.50:3300/0 conn(0x7f112806bc30 0x7f11280b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112cbe1640 1 -- 172.21.15.188:0/2464898286 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2716187494 0 0) 0x7f11280b5ad0 con 0x7f1128070590 2024-04-02T16:21:38.414 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112cbe1640 1 -- 172.21.15.188:0/2464898286 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1114003520 con 0x7f1128070590 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112d3e2640 1 -- 172.21.15.188:0/2464898286 >> v1:172.21.15.188:6789/0 conn(0x7f1128070590 legacy=0x7f11280b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112d3e2640 1 -- 172.21.15.188:0/2464898286 >> v1:172.21.15.50:6789/0 conn(0x7f112815e390 legacy=0x7f112806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112cbe1640 1 -- 172.21.15.188:0/2464898286 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 952042007 0 0) 0x7f11280b5db0 con 0x7f112815e390 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.412+0000 7f112d3e2640 1 -- 172.21.15.188:0/2464898286 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11280b44a0 con 0x7f112806bc30 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.413+0000 7f112d3e2640 1 --2- 172.21.15.188:0/2464898286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f112806bc30 0x7f11280b4ca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f11240097a0 tx=0x7f112402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc75b869f768ff44 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.413+0000 7f112cbe1640 1 -- 172.21.15.188:0/2464898286 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124034070 con 0x7f112806bc30 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.413+0000 7f112cbe1640 1 -- 172.21.15.188:0/2464898286 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1124030610 con 0x7f112806bc30 2024-04-02T16:21:38.415 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.413+0000 7f112cbe1640 1 -- 172.21.15.188:0/2464898286 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240309c0 con 0x7f112806bc30 2024-04-02T16:21:38.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.414+0000 7f112fe47640 1 -- 172.21.15.188:0/2464898286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f112806bc30 msgr2=0x7f11280b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.414+0000 7f112fe47640 1 --2- 172.21.15.188:0/2464898286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f112806bc30 0x7f11280b4ca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f11240097a0 tx=0x7f112402b960 comp rx=0 tx=0).stop 2024-04-02T16:21:38.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.414+0000 7f112fe47640 1 -- 172.21.15.188:0/2464898286 shutdown_connections 2024-04-02T16:21:38.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.414+0000 7f112fe47640 1 --2- 172.21.15.188:0/2464898286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f112806bc30 0x7f11280b4ca0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.414+0000 7f112fe47640 1 -- 172.21.15.188:0/2464898286 >> 172.21.15.188:0/2464898286 conn(0x7f112806e680 msgr2=0x7f112806ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.414+0000 7f112fe47640 1 -- 172.21.15.188:0/2464898286 shutdown_connections 2024-04-02T16:21:38.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.414+0000 7f112fe47640 1 -- 172.21.15.188:0/2464898286 wait complete. 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.415+0000 7f112fe47640 1 Processor -- start 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.415+0000 7f112fe47640 1 -- start start 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112fe47640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f112806bc30 0x7f11280d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112fe47640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1128070590 0x7f112815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112fe47640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f112815e390 0x7f112815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112fe47640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1128169010 con 0x7f1128070590 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112fe47640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1128168e90 con 0x7f112815e390 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112fe47640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1128169190 con 0x7f112806bc30 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f112815e390 0x7f112815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112dbe3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f112806bc30 0x7f11280d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f112815e390 0x7f112815c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:60040/0 (socket says 172.21.15.188:60040) 2024-04-02T16:21:38.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 -- 172.21.15.188:0/2275912880 learned_addr learned my addr 172.21.15.188:0/2275912880 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112d3e2640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1128070590 0x7f112815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 -- 172.21.15.188:0/2275912880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f112806bc30 msgr2=0x7f11280d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f112806bc30 0x7f11280d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 -- 172.21.15.188:0/2275912880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1128070590 msgr2=0x7f112815bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1128070590 0x7f112815bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 -- 172.21.15.188:0/2275912880 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11280affe0 con 0x7f112815e390 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f112e3e4640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f112815e390 0x7f112815c270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f112000cb40 tx=0x7f112000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f111a7fc640 1 -- 172.21.15.188:0/2275912880 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120017070 con 0x7f112815e390 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.416+0000 7f111a7fc640 1 -- 172.21.15.188:0/2275912880 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112000aca0 con 0x7f112815e390 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.417+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112815e910 con 0x7f112815e390 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.417+0000 7f111a7fc640 1 -- 172.21.15.188:0/2275912880 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120012620 con 0x7f112815e390 2024-04-02T16:21:38.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.417+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11281615d0 con 0x7f112815e390 2024-04-02T16:21:38.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.417+0000 7f111a7fc640 1 -- 172.21.15.188:0/2275912880 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f112001a070 con 0x7f112815e390 2024-04-02T16:21:38.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.417+0000 7f111a7fc640 1 -- 172.21.15.188:0/2275912880 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f1120007660 con 0x7f112815e390 2024-04-02T16:21:38.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.417+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11281597a0 con 0x7f112815e390 2024-04-02T16:21:38.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.419+0000 7f111a7fc640 1 -- 172.21.15.188:0/2275912880 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1120021070 con 0x7f112815e390 2024-04-02T16:21:38.494 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.493+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "427efa9c-ad3f-4f2b-8205-f1640d4e55ef", "id": 6} v 0) -- 0x7f11280756f0 con 0x7f112815e390 2024-04-02T16:21:38.498 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.497+0000 7f111a7fc640 1 -- 172.21.15.188:0/2275912880 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "427efa9c-ad3f-4f2b-8205-f1640d4e55ef", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f112001c510 con 0x7f112815e390 2024-04-02T16:21:38.498 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-04-02T16:21:38.499 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f112815e390 msgr2=0x7f112815c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.499 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f112815e390 0x7f112815c270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f112000cb40 tx=0x7f112000a060 comp rx=0 tx=0).stop 2024-04-02T16:21:38.499 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 shutdown_connections 2024-04-02T16:21:38.499 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f112815e390 0x7f112815c270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.499 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1128070590 0x7f112815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.499 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 --2- 172.21.15.188:0/2275912880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f112806bc30 0x7f11280d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 >> 172.21.15.188:0/2275912880 conn(0x7f112806e680 msgr2=0x7f1128072d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 shutdown_connections 2024-04-02T16:21:38.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.498+0000 7f112fe47640 1 -- 172.21.15.188:0/2275912880 wait complete. 2024-04-02T16:21:38.506 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:38.505+0000 7f888a02b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T16:21:38.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 5080b25c-8467-43c2-b078-f006968201f3 7 2024-04-02T16:21:38.545 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:38.544+0000 7f0e82db9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T16:21:38.610 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.609+0000 7ffb0e033640 1 Processor -- start 2024-04-02T16:21:38.610 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.609+0000 7ffb0e033640 1 -- start start 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffb0e033640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb08144a10 0x7ffb08072440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffb0e033640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb080cf9d0 con 0x7ffb081444b0 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffb0e033640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb080cfc70 con 0x7ffb08072980 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffb0e033640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffb08072db0 con 0x7ffb08144a10 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffafffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb08144a10 0x7ffb08072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffafffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb08144a10 0x7ffb08072440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:60048/0 (socket says 172.21.15.188:60048) 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffafffff640 1 -- 172.21.15.188:0/2559248434 learned_addr learned my addr 172.21.15.188:0/2559248434 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.610+0000 7ffb07fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ffb08072980 0x7ffb080b4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:58504/0 (socket says 172.21.15.188:58504) 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.611+0000 7ffafffff640 1 -- 172.21.15.188:0/2559248434 >> v1:172.21.15.188:6789/0 conn(0x7ffb08072980 legacy=0x7ffb080b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.612 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.611+0000 7ffafffff640 1 -- 172.21.15.188:0/2559248434 >> v1:172.21.15.50:6789/0 conn(0x7ffb081444b0 legacy=0x7ffb08070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.613 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.611+0000 7ffafffff640 1 -- 172.21.15.188:0/2559248434 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb080cff50 con 0x7ffb08144a10 2024-04-02T16:21:38.613 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.611+0000 7ffafffff640 1 --2- 172.21.15.188:0/2559248434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb08144a10 0x7ffb08072440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000bb10 tx=0x7ffaf002e1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef58898b7e13fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.613 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb06ffd640 1 -- 172.21.15.188:0/2559248434 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf003b070 con 0x7ffb08144a10 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb06ffd640 1 -- 172.21.15.188:0/2559248434 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf0036440 con 0x7ffb08144a10 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb06ffd640 1 -- 172.21.15.188:0/2559248434 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0036810 con 0x7ffb08144a10 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb0e033640 1 -- 172.21.15.188:0/2559248434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb08144a10 msgr2=0x7ffb08072440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb0e033640 1 --2- 172.21.15.188:0/2559248434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb08144a10 0x7ffb08072440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000bb10 tx=0x7ffaf002e1f0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb0e033640 1 -- 172.21.15.188:0/2559248434 shutdown_connections 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb0e033640 1 --2- 172.21.15.188:0/2559248434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb08144a10 0x7ffb08072440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb0e033640 1 -- 172.21.15.188:0/2559248434 >> 172.21.15.188:0/2559248434 conn(0x7ffb0806e680 msgr2=0x7ffb0806ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb0e033640 1 -- 172.21.15.188:0/2559248434 shutdown_connections 2024-04-02T16:21:38.614 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.612+0000 7ffb0e033640 1 -- 172.21.15.188:0/2559248434 wait complete. 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.613+0000 7ffb0e033640 1 Processor -- start 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.613+0000 7ffb0e033640 1 -- start start 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb0e033640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb08072980 0x7ffb0813af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb0e033640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb081444b0 0x7ffb0815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb0e033640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb08144a10 0x7ffb080b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb0e033640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb080d0d10 con 0x7ffb08144a10 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb0e033640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb080d0510 con 0x7ffb081444b0 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb0e033640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffb080b4b30 con 0x7ffb08072980 2024-04-02T16:21:38.617 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffafffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb081444b0 0x7ffb0815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffafffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb081444b0 0x7ffb0815c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:60058/0 (socket says 172.21.15.188:60058) 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffafffff640 1 -- 172.21.15.188:0/2706832222 learned_addr learned my addr 172.21.15.188:0/2706832222 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb077fe640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb08072980 0x7ffb0813af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.614+0000 7ffb07fff640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb08144a10 0x7ffb080b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffafffff640 1 -- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb08072980 msgr2=0x7ffb0813af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffafffff640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb08072980 0x7ffb0813af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffafffff640 1 -- 172.21.15.188:0/2706832222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb08144a10 msgr2=0x7ffb080b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffafffff640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb08144a10 0x7ffb080b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffafffff640 1 -- 172.21.15.188:0/2706832222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0815d1b0 con 0x7ffb081444b0 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb077fe640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb08072980 0x7ffb0813af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffafffff640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb081444b0 0x7ffb0815c6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ffaf002e230 tx=0x7ffaf00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb04ff9640 1 -- 172.21.15.188:0/2706832222 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf003b040 con 0x7ffb081444b0 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb04ff9640 1 -- 172.21.15.188:0/2706832222 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf0046070 con 0x7ffb081444b0 2024-04-02T16:21:38.618 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0815d490 con 0x7ffb081444b0 2024-04-02T16:21:38.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb04ff9640 1 -- 172.21.15.188:0/2706832222 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0034bb0 con 0x7ffb081444b0 2024-04-02T16:21:38.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0815aad0 con 0x7ffb081444b0 2024-04-02T16:21:38.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb04ff9640 1 -- 172.21.15.188:0/2706832222 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffaf004b460 con 0x7ffb081444b0 2024-04-02T16:21:38.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.615+0000 7ffb04ff9640 1 -- 172.21.15.188:0/2706832222 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ffaf0041aa0 con 0x7ffb081444b0 2024-04-02T16:21:38.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.616+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb080756f0 con 0x7ffb081444b0 2024-04-02T16:21:38.619 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.618+0000 7ffb04ff9640 1 -- 172.21.15.188:0/2706832222 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ffaf0041d40 con 0x7ffb081444b0 2024-04-02T16:21:38.649 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:38.648+0000 7f888a02b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T16:21:38.686 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:38.685+0000 7f0e82db9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T16:21:38.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.690+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5080b25c-8467-43c2-b078-f006968201f3", "id": 7} v 0) -- 0x7ffb0815d660 con 0x7ffb081444b0 2024-04-02T16:21:38.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.694+0000 7ffb04ff9640 1 -- 172.21.15.188:0/2706832222 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5080b25c-8467-43c2-b078-f006968201f3", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ffaf0041440 con 0x7ffb081444b0 2024-04-02T16:21:38.695 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-04-02T16:21:38.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb081444b0 msgr2=0x7ffb0815c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb081444b0 0x7ffb0815c6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ffaf002e230 tx=0x7ffaf00046c0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 shutdown_connections 2024-04-02T16:21:38.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb08144a10 0x7ffb080b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb081444b0 0x7ffb0815c6f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 --2- 172.21.15.188:0/2706832222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb08072980 0x7ffb0813af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 >> 172.21.15.188:0/2706832222 conn(0x7ffb0806e680 msgr2=0x7ffb08141280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 shutdown_connections 2024-04-02T16:21:38.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:21:38.695+0000 7ffb0e033640 1 -- 172.21.15.188:0/2706832222 wait complete. 2024-04-02T16:21:38.707 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-02T16:21:38.707 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-02T16:21:38.709 INFO:tasks.ceph.osd.0:Started 2024-04-02T16:21:38.709 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-02T16:21:38.709 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-02T16:21:38.714 INFO:tasks.ceph.osd.1:Started 2024-04-02T16:21:38.714 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-02T16:21:38.714 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-02T16:21:38.716 INFO:tasks.ceph.osd.2:Started 2024-04-02T16:21:38.716 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-02T16:21:38.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-02T16:21:38.718 INFO:tasks.ceph.osd.3:Started 2024-04-02T16:21:38.718 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-02T16:21:38.718 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-02T16:21:38.720 INFO:tasks.ceph.osd.4:Started 2024-04-02T16:21:38.720 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-02T16:21:38.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-02T16:21:38.722 INFO:tasks.ceph.osd.5:Started 2024-04-02T16:21:38.722 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-02T16:21:38.723 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-02T16:21:38.724 INFO:tasks.ceph.osd.6:Started 2024-04-02T16:21:38.725 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-02T16:21:38.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-02T16:21:38.727 INFO:tasks.ceph.osd.7:Started 2024-04-02T16:21:38.727 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:21:38.784 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:38.783+0000 7f888a02b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T16:21:38.837 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:38.836+0000 7f888a02b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T16:21:38.837 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:38.836+0000 7f0e82db9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T16:21:38.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.846+0000 7f4ad875f640 1 Processor -- start 2024-04-02T16:21:38.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.846+0000 7f4ad875f640 1 -- start start 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad875f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ad00d0da0 0x7f4ad00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad875f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ad00b46c0 0x7f4ad00cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad875f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad00ccba0 con 0x7f4ad015c330 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad875f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad00d01c0 con 0x7f4ad00d0da0 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad875f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad00cd770 con 0x7f4ad00b46c0 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad5cfa640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ad00d0da0 0x7f4ad00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad5cfa640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ad00d0da0 0x7f4ad00cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52154/0 (socket says 172.21.15.50:52154) 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad5cfa640 1 -- 172.21.15.50:0/2819575143 learned_addr learned my addr 172.21.15.50:0/2819575143 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad6cfc640 1 --2- 172.21.15.50:0/2819575143 >> v2:172.21.15.188:3300/0 conn(0x7f4ad00b46c0 0x7f4ad00cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad5cfa640 1 -- 172.21.15.50:0/2819575143 >> v2:172.21.15.188:3300/0 conn(0x7f4ad00b46c0 msgr2=0x7f4ad00cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:38.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad5cfa640 1 --2- 172.21.15.50:0/2819575143 >> v2:172.21.15.188:3300/0 conn(0x7f4ad00b46c0 0x7f4ad00cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad5cfa640 1 -- 172.21.15.50:0/2819575143 >> v1:172.21.15.188:6789/0 conn(0x7f4ad015c330 legacy=0x7f4ad00d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.847+0000 7f4ad5cfa640 1 -- 172.21.15.50:0/2819575143 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad00cce40 con 0x7f4ad00d0da0 2024-04-02T16:21:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.848+0000 7f4ad5cfa640 1 --2- 172.21.15.50:0/2819575143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00d0da0 0x7f4ad00cdc00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00d1ff0 tx=0x7f4acc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e141fffbe3202fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.848+0000 7f4ad54f9640 1 -- 172.21.15.50:0/2819575143 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc034070 con 0x7f4ad00d0da0 2024-04-02T16:21:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.848+0000 7f4ad54f9640 1 -- 172.21.15.50:0/2819575143 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc030860 con 0x7f4ad00d0da0 2024-04-02T16:21:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad54f9640 1 -- 172.21.15.50:0/2819575143 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc030c30 con 0x7f4ad00d0da0 2024-04-02T16:21:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 -- 172.21.15.50:0/2819575143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00d0da0 msgr2=0x7f4ad00cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 --2- 172.21.15.50:0/2819575143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00d0da0 0x7f4ad00cdc00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00d1ff0 tx=0x7f4acc02b9a0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 -- 172.21.15.50:0/2819575143 shutdown_connections 2024-04-02T16:21:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 --2- 172.21.15.50:0/2819575143 >> v2:172.21.15.188:3300/0 conn(0x7f4ad00b46c0 0x7f4ad00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 --2- 172.21.15.50:0/2819575143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00d0da0 0x7f4ad00cdc00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 -- 172.21.15.50:0/2819575143 >> 172.21.15.50:0/2819575143 conn(0x7f4ad00b3ba0 msgr2=0x7f4ad00b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 -- 172.21.15.50:0/2819575143 shutdown_connections 2024-04-02T16:21:38.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.849+0000 7f4ad875f640 1 -- 172.21.15.50:0/2819575143 wait complete. 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.851+0000 7f4ad875f640 1 Processor -- start 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.851+0000 7f4ad875f640 1 -- start start 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.851+0000 7f4ad875f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00b46c0 0x7f4ad014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.851+0000 7f4ad875f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad00d0da0 0x7f4ad015ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.851+0000 7f4ad875f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad015c330 0x7f4ad00b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.851+0000 7f4ad875f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad006f610 con 0x7f4ad00b46c0 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad875f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad00cd400 con 0x7f4ad00d0da0 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00b46c0 0x7f4ad014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00b46c0 0x7f4ad014a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52160/0 (socket says 172.21.15.50:52160) 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 -- 172.21.15.50:0/2914566095 learned_addr learned my addr 172.21.15.50:0/2914566095 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad006f790 con 0x7f4ad015c330 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad6cfc640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad015c330 0x7f4ad00b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad5cfa640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad00d0da0 0x7f4ad015ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 -- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad015c330 msgr2=0x7f4ad00b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad015c330 0x7f4ad00b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 -- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad00d0da0 msgr2=0x7f4ad015ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad00d0da0 0x7f4ad015ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 -- 172.21.15.50:0/2914566095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad014aa10 con 0x7f4ad00b46c0 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad6cfc640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad015c330 0x7f4ad00b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad64fb640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00b46c0 0x7f4ad014a4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c3c0 tx=0x7f4ac000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ac6ffd640 1 -- 172.21.15.50:0/2914566095 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00052d0 con 0x7f4ad00b46c0 2024-04-02T16:21:38.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ac6ffd640 1 -- 172.21.15.50:0/2914566095 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac0005490 con 0x7f4ad00b46c0 2024-04-02T16:21:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.852+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad015d420 con 0x7f4ad00b46c0 2024-04-02T16:21:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.853+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad015e350 con 0x7f4ad00b46c0 2024-04-02T16:21:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.853+0000 7f4ac6ffd640 1 -- 172.21.15.50:0/2914566095 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0007260 con 0x7f4ad00b46c0 2024-04-02T16:21:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.853+0000 7f4ac6ffd640 1 -- 172.21.15.50:0/2914566095 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ac0007400 con 0x7f4ad00b46c0 2024-04-02T16:21:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.853+0000 7f4ac6ffd640 1 -- 172.21.15.50:0/2914566095 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4ac0011c60 con 0x7f4ad00b46c0 2024-04-02T16:21:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.853+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad00756f0 con 0x7f4ad00b46c0 2024-04-02T16:21:38.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.856+0000 7f4ac6ffd640 1 -- 172.21.15.50:0/2914566095 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4ad014aa10 con 0x7f4ad00b46c0 2024-04-02T16:21:38.887 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:38.886+0000 7f0e82db9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T16:21:38.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.917+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ad00b8030 con 0x7f4ad00b46c0 2024-04-02T16:21:38.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.918+0000 7f4ac6ffd640 1 -- 172.21.15.50:0/2914566095 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4ac00035f0 con 0x7f4ad00b46c0 2024-04-02T16:21:38.919 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:38.919 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":10,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:21:38.692635+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:21:38.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00b46c0 msgr2=0x7f4ad014a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:38.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00b46c0 0x7f4ad014a4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c3c0 tx=0x7f4ac000c880 comp rx=0 tx=0).stop 2024-04-02T16:21:38.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 shutdown_connections 2024-04-02T16:21:38.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ad015c330 0x7f4ad00b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ad00d0da0 0x7f4ad015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 --2- 172.21.15.50:0/2914566095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ad00b46c0 0x7f4ad014a4d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:38.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 >> 172.21.15.50:0/2914566095 conn(0x7f4ad00b3ba0 msgr2=0x7f4ad00aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:38.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 shutdown_connections 2024-04-02T16:21:38.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:38.919+0000 7f4ad875f640 1 -- 172.21.15.50:0/2914566095 wait complete. 2024-04-02T16:21:38.930 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-02T16:21:38.930 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-02T16:21:38.950 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:38.949+0000 7f888a02b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:21:39.006 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:39.005+0000 7f0e82db9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:21:39.043 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:39.042+0000 7f888a02b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T16:21:39.106 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:39.105+0000 7f0e82db9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T16:21:39.198 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:39.197+0000 7f888a02b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:21:39.240 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:39.239+0000 7f888a02b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T16:21:39.269 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:39.268+0000 7f0e82db9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:21:39.313 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:39.312+0000 7f0e82db9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T16:21:39.318 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T16:21:39.317+0000 7ff98b012640 -1 Falling back to public interface 2024-04-02T16:21:39.323 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T16:21:39.322+0000 7f0533fe1640 -1 Falling back to public interface 2024-04-02T16:21:39.325 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T16:21:39.325+0000 7f2266218640 -1 Falling back to public interface 2024-04-02T16:21:39.329 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T16:21:39.328+0000 7f1ff6e19640 -1 Falling back to public interface 2024-04-02T16:21:39.330 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T16:21:39.328+0000 7f1daacaa640 -1 Falling back to public interface 2024-04-02T16:21:39.330 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T16:21:39.329+0000 7f1da9c59640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-04-02T16:21:39.342 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T16:21:39.340+0000 7f9ffde15640 -1 Falling back to public interface 2024-04-02T16:21:39.343 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T16:21:39.341+0000 7fde4f40f640 -1 Falling back to public interface 2024-04-02T16:21:39.349 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T16:21:39.348+0000 7fd248417640 -1 Falling back to public interface 2024-04-02T16:21:39.397 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:39.396+0000 7f888a02b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T16:21:39.481 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:39.479+0000 7f0e82db9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T16:21:39.736 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T16:21:39.827 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.826+0000 7f767d196640 1 Processor -- start 2024-04-02T16:21:39.827 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.826+0000 7f767d196640 1 -- start start 2024-04-02T16:21:39.830 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.826+0000 7f767d196640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f767815a8a0 0x7f76780cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:39.830 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.826+0000 7f767d196640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767815ac80 con 0x7f76780b49e0 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.826+0000 7f767d196640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780b5ad0 con 0x7f767806f2e0 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.826+0000 7f767d196640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f76780b4ec0 con 0x7f767815a8a0 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f7676d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f767815a8a0 0x7f76780cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f766ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f76780b49e0 0x7f767806ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:40008/0 (socket says 172.21.15.50:40008) 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f7676d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f767815a8a0 0x7f76780cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52170/0 (socket says 172.21.15.50:52170) 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f766ffff640 1 -- 172.21.15.50:0/2127138951 learned_addr learned my addr 172.21.15.50:0/2127138951 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f7676d9d640 1 -- 172.21.15.50:0/2127138951 >> v1:172.21.15.188:6789/0 conn(0x7f767806f2e0 legacy=0x7f767806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f7676d9d640 1 -- 172.21.15.50:0/2127138951 >> v1:172.21.15.50:6789/0 conn(0x7f76780b49e0 legacy=0x7f767806ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f7676d9d640 1 -- 172.21.15.50:0/2127138951 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76780b51a0 con 0x7f767815a8a0 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.827+0000 7f767659c640 1 -- 172.21.15.50:0/2127138951 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460808932 0 0) 0x7f767815ac80 con 0x7f76780b49e0 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f7676d9d640 1 --2- 172.21.15.50:0/2127138951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767815a8a0 0x7f76780cfe30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f766800a940 tx=0x7f766802bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6051603b7bf310e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:39.831 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f767659c640 1 -- 172.21.15.50:0/2127138951 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668039070 con 0x7f767815a8a0 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f767659c640 1 -- 172.21.15.50:0/2127138951 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766802cad0 con 0x7f767815a8a0 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f767659c640 1 -- 172.21.15.50:0/2127138951 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766800b040 con 0x7f767815a8a0 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f767d196640 1 -- 172.21.15.50:0/2127138951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767815a8a0 msgr2=0x7f76780cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f767d196640 1 --2- 172.21.15.50:0/2127138951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767815a8a0 0x7f76780cfe30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f766800a940 tx=0x7f766802bd20 comp rx=0 tx=0).stop 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f767d196640 1 -- 172.21.15.50:0/2127138951 shutdown_connections 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.828+0000 7f767d196640 1 --2- 172.21.15.50:0/2127138951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767815a8a0 0x7f76780cfe30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.829+0000 7f767d196640 1 -- 172.21.15.50:0/2127138951 >> 172.21.15.50:0/2127138951 conn(0x7f76780d0850 msgr2=0x7f76780d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.829+0000 7f767d196640 1 -- 172.21.15.50:0/2127138951 shutdown_connections 2024-04-02T16:21:39.832 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.829+0000 7f767d196640 1 -- 172.21.15.50:0/2127138951 wait complete. 2024-04-02T16:21:39.835 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.830+0000 7f767d196640 1 Processor -- start 2024-04-02T16:21:39.835 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.830+0000 7f767d196640 1 -- start start 2024-04-02T16:21:39.835 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f767d196640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767806f2e0 0x7f76780ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:39.835 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f767d196640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76780b49e0 0x7f7678147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f767d196640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f767815a8a0 0x7f76780b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f767d196640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76780d0370 con 0x7f767806f2e0 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f767d196640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76780b5760 con 0x7f76780b49e0 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f767d196640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f76780d04f0 con 0x7f767815a8a0 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767806f2e0 0x7f76780ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767806f2e0 0x7f76780ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52178/0 (socket says 172.21.15.50:52178) 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 -- 172.21.15.50:0/193016183 learned_addr learned my addr 172.21.15.50:0/193016183 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f766ffff640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76780b49e0 0x7f7678147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f767759e640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f767815a8a0 0x7f76780b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 -- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f767815a8a0 msgr2=0x7f76780b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f767815a8a0 0x7f76780b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 -- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76780b49e0 msgr2=0x7f7678147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76780b49e0 0x7f7678147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:39.836 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 -- 172.21.15.50:0/193016183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76780ab840 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f7676d9d640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767806f2e0 0x7f76780ab300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7668037040 tx=0x7f76680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.831+0000 7f766f7fe640 1 -- 172.21.15.50:0/193016183 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668043030 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.832+0000 7f766f7fe640 1 -- 172.21.15.50:0/193016183 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766802c8d0 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.832+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767815d3f0 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.832+0000 7f766f7fe640 1 -- 172.21.15.50:0/193016183 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668039040 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.832+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767815d9b0 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.832+0000 7f766f7fe640 1 -- 172.21.15.50:0/193016183 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7668046070 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.832+0000 7f766f7fe640 1 -- 172.21.15.50:0/193016183 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7668048ae0 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.832+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76780750e0 con 0x7f767806f2e0 2024-04-02T16:21:39.837 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.835+0000 7f766f7fe640 1 -- 172.21.15.50:0/193016183 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f766802caa0 con 0x7f767806f2e0 2024-04-02T16:21:39.898 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.897+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76780aba70 con 0x7f767806f2e0 2024-04-02T16:21:39.899 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.898+0000 7f766f7fe640 1 -- 172.21.15.50:0/193016183 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7668048da0 con 0x7f767806f2e0 2024-04-02T16:21:39.900 INFO:teuthology.misc.health.smithi050.stdout: 2024-04-02T16:21:39.900 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":10,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:21:38.692635+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767806f2e0 msgr2=0x7f76780ab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767806f2e0 0x7f76780ab300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7668037040 tx=0x7f76680046c0 comp rx=0 tx=0).stop 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 shutdown_connections 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f767815a8a0 0x7f76780b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76780b49e0 0x7f7678147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 --2- 172.21.15.50:0/193016183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f767806f2e0 0x7f76780ab300 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 >> 172.21.15.50:0/193016183 conn(0x7f76780d0850 msgr2=0x7f76780b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 shutdown_connections 2024-04-02T16:21:39.901 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:39.899+0000 7f767d196640 1 -- 172.21.15.50:0/193016183 wait complete. 2024-04-02T16:21:39.909 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-02T16:21:40.212 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:40.211+0000 7f888a02b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T16:21:40.342 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:40.341+0000 7f0e82db9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T16:21:40.488 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:40.487+0000 7f888a02b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T16:21:40.539 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:40.539+0000 7f888a02b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T16:21:40.636 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:40.634+0000 7f0e82db9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T16:21:40.655 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:40.654+0000 7f888a02b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T16:21:40.692 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:40.691+0000 7f0e82db9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T16:21:40.813 INFO:tasks.ceph.mgr.y.smithi050.stderr:2024-04-02T16:21:40.812+0000 7f0e82db9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T16:21:41.671 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:41.670+0000 7f882a426640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-02T16:21:41.672 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-02T16:21:41.671+0000 7f882a426640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-02T16:21:42.812 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T16:21:42.811+0000 7ff98b012640 -1 osd.7 0 log_to_monitors true 2024-04-02T16:21:42.818 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T16:21:42.815+0000 7f0533fe1640 -1 osd.6 0 log_to_monitors true 2024-04-02T16:21:42.826 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T16:21:42.825+0000 7f1daacaa640 -1 osd.1 0 log_to_monitors true 2024-04-02T16:21:42.827 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T16:21:42.826+0000 7f2266218640 -1 osd.3 0 log_to_monitors true 2024-04-02T16:21:42.853 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T16:21:42.852+0000 7f9ffde15640 -1 osd.5 0 log_to_monitors true 2024-04-02T16:21:42.854 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T16:21:42.854+0000 7fd248417640 -1 osd.4 0 log_to_monitors true 2024-04-02T16:21:42.891 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T16:21:42.890+0000 7fde4f40f640 -1 osd.2 0 log_to_monitors true 2024-04-02T16:21:42.898 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T16:21:42.897+0000 7f1ff6e19640 -1 osd.0 0 log_to_monitors true 2024-04-02T16:21:46.076 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T16:21:46.074+0000 7fde4ad3a640 -1 osd.2 0 waiting for initial osdmap 2024-04-02T16:21:46.097 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T16:21:46.095+0000 7fde42b5a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.102 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T16:21:46.100+0000 7f2261b85640 -1 osd.3 0 waiting for initial osdmap 2024-04-02T16:21:46.112 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T16:21:46.111+0000 7f22599a5640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.122 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T16:21:46.121+0000 7f1da6c40640 -1 osd.1 0 waiting for initial osdmap 2024-04-02T16:21:46.141 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T16:21:46.140+0000 7f1d9ea60640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.146 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T16:21:46.145+0000 7f1ff27ec640 -1 osd.0 0 waiting for initial osdmap 2024-04-02T16:21:46.158 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T16:21:46.157+0000 7f1fea60c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.715 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T16:21:46.713+0000 7f052ff77640 -1 osd.6 0 waiting for initial osdmap 2024-04-02T16:21:46.715 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T16:21:46.736 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T16:21:46.735+0000 7f0527d97640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.743 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T16:21:46.742+0000 7fd243dec640 -1 osd.4 0 waiting for initial osdmap 2024-04-02T16:21:46.749 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T16:21:46.748+0000 7ff986959640 -1 osd.7 0 waiting for initial osdmap 2024-04-02T16:21:46.755 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T16:21:46.755+0000 7fd23bc0c640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.760 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T16:21:46.760+0000 7ff97e779640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.788 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T16:21:46.788+0000 7f9ffa091640 -1 osd.5 0 waiting for initial osdmap 2024-04-02T16:21:46.798 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T16:21:46.797+0000 7f9ff169d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:21:46.811 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.810+0000 7ffbaa660640 1 Processor -- start 2024-04-02T16:21:46.812 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.810+0000 7ffbaa660640 1 -- start start 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffbaa660640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffba415e460 0x7ffba40d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffbaa660640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffba40d3980 0x7ffba40b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffbaa660640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba415e840 con 0x7ffba40d1210 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffbaa660640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ffba40b7960 con 0x7ffba40d3980 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffbaa660640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffba41623b0 con 0x7ffba415e460 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffba40d3980 0x7ffba40b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba8bfd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ffba40d1210 0x7ffba40d2840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44272/0 (socket says 172.21.15.50:44272) 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba8bfd640 1 -- 172.21.15.50:0/2805505934 learned_addr learned my addr 172.21.15.50:0/2805505934 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba3fff640 1 --2- 172.21.15.50:0/2805505934 >> v2:172.21.15.188:3300/0 conn(0x7ffba415e460 0x7ffba40d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:46.813 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba37fe640 1 -- 172.21.15.50:0/2805505934 >> v2:172.21.15.188:3300/0 conn(0x7ffba415e460 msgr2=0x7ffba40d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:46.814 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba37fe640 1 --2- 172.21.15.50:0/2805505934 >> v2:172.21.15.188:3300/0 conn(0x7ffba415e460 0x7ffba40d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:46.814 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba37fe640 1 -- 172.21.15.50:0/2805505934 >> v1:172.21.15.50:6789/0 conn(0x7ffba40d1210 legacy=0x7ffba40d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:46.814 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.811+0000 7ffba2ffd640 1 -- 172.21.15.50:0/2805505934 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1529421856 0 0) 0x7ffba415e840 con 0x7ffba40d1210 2024-04-02T16:21:46.814 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.812+0000 7ffba37fe640 1 -- 172.21.15.50:0/2805505934 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba4162690 con 0x7ffba40d3980 2024-04-02T16:21:46.815 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.812+0000 7ffba37fe640 1 --2- 172.21.15.50:0/2805505934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba40d3980 0x7ffba40b7420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffba40d0e70 tx=0x7ffb8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd003f8fc29bdeaf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:46.815 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.812+0000 7ffba2ffd640 1 -- 172.21.15.50:0/2805505934 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c035020 con 0x7ffba40d3980 2024-04-02T16:21:46.815 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.812+0000 7ffba2ffd640 1 -- 172.21.15.50:0/2805505934 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c031830 con 0x7ffba40d3980 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.812+0000 7ffba2ffd640 1 -- 172.21.15.50:0/2805505934 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c031c00 con 0x7ffba40d3980 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 -- 172.21.15.50:0/2805505934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba40d3980 msgr2=0x7ffba40b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 --2- 172.21.15.50:0/2805505934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba40d3980 0x7ffba40b7420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffba40d0e70 tx=0x7ffb8c02b8d0 comp rx=0 tx=0).stop 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 -- 172.21.15.50:0/2805505934 shutdown_connections 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 --2- 172.21.15.50:0/2805505934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba40d3980 0x7ffba40b7420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 --2- 172.21.15.50:0/2805505934 >> v2:172.21.15.188:3300/0 conn(0x7ffba415e460 0x7ffba40d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 -- 172.21.15.50:0/2805505934 >> 172.21.15.50:0/2805505934 conn(0x7ffba4072dd0 msgr2=0x7ffba40731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 -- 172.21.15.50:0/2805505934 shutdown_connections 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.813+0000 7ffbaa660640 1 -- 172.21.15.50:0/2805505934 wait complete. 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.814+0000 7ffbaa660640 1 Processor -- start 2024-04-02T16:21:46.816 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.814+0000 7ffbaa660640 1 -- start start 2024-04-02T16:21:46.818 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffbaa660640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d1210 0x7ffba4154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:46.818 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffbaa660640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d3980 0x7ffba41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:46.818 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffbaa660640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba415e460 0x7ffba415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:46.818 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffbaa660640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4161fb0 con 0x7ffba415e460 2024-04-02T16:21:46.818 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffbaa660640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4162c50 con 0x7ffba40d1210 2024-04-02T16:21:46.818 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffbaa660640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba4162130 con 0x7ffba40d3980 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffba8bfd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba415e460 0x7ffba415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffba8bfd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba415e460 0x7ffba415cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55204/0 (socket says 172.21.15.50:55204) 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffba8bfd640 1 -- 172.21.15.50:0/1388381889 learned_addr learned my addr 172.21.15.50:0/1388381889 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffba3fff640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d1210 0x7ffba4154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.815+0000 7ffba37fe640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d3980 0x7ffba41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba8bfd640 1 -- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d3980 msgr2=0x7ffba41599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba8bfd640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d3980 0x7ffba41599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba8bfd640 1 -- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d1210 msgr2=0x7ffba4154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba8bfd640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d1210 0x7ffba4154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba8bfd640 1 -- 172.21.15.50:0/1388381889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba4158600 con 0x7ffba415e460 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba3fff640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d1210 0x7ffba4154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba37fe640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d3980 0x7ffba41599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba8bfd640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba415e460 0x7ffba415cbd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00cb20 tx=0x7ffb9c0021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:46.819 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba0ff9640 1 -- 172.21.15.50:0/1388381889 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00ac00 con 0x7ffba415e460 2024-04-02T16:21:46.820 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba0ff9640 1 -- 172.21.15.50:0/1388381889 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c00adc0 con 0x7ffba415e460 2024-04-02T16:21:46.820 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffba0ff9640 1 -- 172.21.15.50:0/1388381889 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c012640 con 0x7ffba415e460 2024-04-02T16:21:46.820 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba41588e0 con 0x7ffba415e460 2024-04-02T16:21:46.820 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.816+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba4159490 con 0x7ffba415e460 2024-04-02T16:21:46.820 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.817+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba40750e0 con 0x7ffba415e460 2024-04-02T16:21:46.821 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.818+0000 7ffba0ff9640 1 -- 172.21.15.50:0/1388381889 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ffb9c0127e0 con 0x7ffba415e460 2024-04-02T16:21:46.821 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.818+0000 7ffba0ff9640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8406c860 0x7ffb8406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:46.821 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.818+0000 7ffba0ff9640 1 -- 172.21.15.50:0/1388381889 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7ffb9c08b260 con 0x7ffba415e460 2024-04-02T16:21:46.821 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.818+0000 7ffba3fff640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8406c860 0x7ffb8406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:46.821 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.819+0000 7ffba3fff640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8406c860 0x7ffb8406ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009b90 tx=0x7ffb94008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:46.824 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:46.823+0000 7ffba0ff9640 1 -- 172.21.15.50:0/1388381889 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffb9c05a590 con 0x7ffba415e460 2024-04-02T16:21:47.012 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.011+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffba40d3e90 con 0x7ffba415e460 2024-04-02T16:21:47.012 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.011+0000 7ffba0ff9640 1 -- 172.21.15.50:0/1388381889 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7ffb9c059e90 con 0x7ffba415e460 2024-04-02T16:21:47.013 INFO:teuthology.misc.health.smithi050.stdout: 2024-04-02T16:21:47.013 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":12,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:21:44.673810+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:21:47.014 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8406c860 msgr2=0x7ffb8406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8406c860 0x7ffb8406ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009b90 tx=0x7ffb94008040 comp rx=0 tx=0).stop 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba415e460 msgr2=0x7ffba415cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba415e460 0x7ffba415cbd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00cb20 tx=0x7ffb9c0021c0 comp rx=0 tx=0).stop 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 shutdown_connections 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8406c860 0x7ffb8406ed20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffba415e460 0x7ffba415cbd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d3980 0x7ffba41599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 --2- 172.21.15.50:0/1388381889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d1210 0x7ffba4154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 >> 172.21.15.50:0/1388381889 conn(0x7ffba4072dd0 msgr2=0x7ffba41566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 shutdown_connections 2024-04-02T16:21:47.015 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:47.013+0000 7ffbaa660640 1 -- 172.21.15.50:0/1388381889 wait complete. 2024-04-02T16:21:47.024 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-02T16:21:53.832 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T16:21:53.934 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.934+0000 7f1d6c6d6640 1 Processor -- start 2024-04-02T16:21:53.934 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.934+0000 7f1d6c6d6640 1 -- start start 2024-04-02T16:21:53.936 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6c6d6640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1d6415e460 0x7f1d640d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:53.936 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6c6d6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1d640d1210 0x7f1d640d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:53.936 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6c6d6640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d6415e840 con 0x7f1d640d3980 2024-04-02T16:21:53.936 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6c6d6640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f1d640d1bc0 con 0x7f1d6415e460 2024-04-02T16:21:53.936 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6c6d6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1d641623b0 con 0x7f1d640d1210 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6a472640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1d6415e460 0x7f1d640d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6a472640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1d6415e460 0x7f1d640d3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55218/0 (socket says 172.21.15.50:55218) 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d69c71640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1d640d3980 0x7f1d640b7420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44288/0 (socket says 172.21.15.50:44288) 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6a472640 1 -- 172.21.15.50:0/2231665759 learned_addr learned my addr 172.21.15.50:0/2231665759 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6ac73640 1 --2- 172.21.15.50:0/2231665759 >> v2:172.21.15.188:3300/0 conn(0x7f1d640d1210 0x7f1d640d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6a472640 1 -- 172.21.15.50:0/2231665759 >> v2:172.21.15.188:3300/0 conn(0x7f1d640d1210 msgr2=0x7f1d640d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6a472640 1 --2- 172.21.15.50:0/2231665759 >> v2:172.21.15.188:3300/0 conn(0x7f1d640d1210 0x7f1d640d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6a472640 1 -- 172.21.15.50:0/2231665759 >> v1:172.21.15.50:6789/0 conn(0x7f1d640d3980 legacy=0x7f1d640b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d69470640 1 -- 172.21.15.50:0/2231665759 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354390155 0 0) 0x7f1d6415e840 con 0x7f1d640d3980 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6a472640 1 -- 172.21.15.50:0/2231665759 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64162690 con 0x7f1d6415e460 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.935+0000 7f1d6ac73640 1 --2- 172.21.15.50:0/2231665759 >> v2:172.21.15.188:3300/0 conn(0x7f1d640d1210 0x7f1d640d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.936+0000 7f1d6a472640 1 --2- 172.21.15.50:0/2231665759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d6415e460 0x7f1d640d3e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d640b6270 tx=0x7f1d6002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c117a10f684291e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.936+0000 7f1d69470640 1 -- 172.21.15.50:0/2231665759 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60034070 con 0x7f1d6415e460 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.936+0000 7f1d69470640 1 -- 172.21.15.50:0/2231665759 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60030860 con 0x7f1d6415e460 2024-04-02T16:21:53.937 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.936+0000 7f1d69470640 1 -- 172.21.15.50:0/2231665759 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60030c30 con 0x7f1d6415e460 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2231665759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d6415e460 msgr2=0x7f1d640d3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2231665759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d6415e460 0x7f1d640d3e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d640b6270 tx=0x7f1d6002b9a0 comp rx=0 tx=0).stop 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2231665759 shutdown_connections 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2231665759 >> v2:172.21.15.188:3300/0 conn(0x7f1d640d1210 0x7f1d640d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2231665759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d6415e460 0x7f1d640d3e40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2231665759 >> 172.21.15.50:0/2231665759 conn(0x7f1d64072dd0 msgr2=0x7f1d640731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2231665759 shutdown_connections 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.937+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2231665759 wait complete. 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.938+0000 7f1d6c6d6640 1 Processor -- start 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.938+0000 7f1d6c6d6640 1 -- start start 2024-04-02T16:21:53.941 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6c6d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d640d1210 0x7f1d640ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6c6d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d640d3980 0x7f1d64157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6c6d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6415e460 0x7f1d64158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6c6d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64161fb0 con 0x7f1d640d1210 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6c6d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64162c50 con 0x7f1d6415e460 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6c6d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d64162130 con 0x7f1d640d3980 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d640d1210 0x7f1d640ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d640d1210 0x7f1d640ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55230/0 (socket says 172.21.15.50:55230) 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 -- 172.21.15.50:0/2702078298 learned_addr learned my addr 172.21.15.50:0/2702078298 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d69c71640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d640d3980 0x7f1d64157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 -- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d640d3980 msgr2=0x7f1d64157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d640d3980 0x7f1d64157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6ac73640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6415e460 0x7f1d64158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 -- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6415e460 msgr2=0x7f1d64158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:53.942 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6415e460 0x7f1d64158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 -- 172.21.15.50:0/2702078298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d640cc650 con 0x7f1d640d1210 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d69c71640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d640d3980 0x7f1d64157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6ac73640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6415e460 0x7f1d64158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.939+0000 7f1d6a472640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d640d1210 0x7f1d640ca710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1d60000c00 tx=0x7f1d60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.940+0000 7f1d5affd640 1 -- 172.21.15.50:0/2702078298 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60048070 con 0x7f1d640d1210 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.940+0000 7f1d5affd640 1 -- 172.21.15.50:0/2702078298 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60039d90 con 0x7f1d640d1210 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.940+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6415aae0 con 0x7f1d640d1210 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.940+0000 7f1d5affd640 1 -- 172.21.15.50:0/2702078298 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60034040 con 0x7f1d640d1210 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.940+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6415adc0 con 0x7f1d640d1210 2024-04-02T16:21:53.943 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.941+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d64067eb0 con 0x7f1d640d1210 2024-04-02T16:21:53.944 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.941+0000 7f1d5affd640 1 -- 172.21.15.50:0/2702078298 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d6004b070 con 0x7f1d640d1210 2024-04-02T16:21:53.944 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.941+0000 7f1d5affd640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1d4806c8f0 0x7f1d4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:53.944 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.942+0000 7f1d5affd640 1 -- 172.21.15.50:0/2702078298 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1d600adab0 con 0x7f1d640d1210 2024-04-02T16:21:53.944 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.942+0000 7f1d69c71640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1d4806c8f0 0x7f1d4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:53.944 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.942+0000 7f1d69c71640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1d4806c8f0 0x7f1d4806edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d6415ec00 tx=0x7f1d54009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:53.946 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:53.946+0000 7f1d5affd640 1 -- 172.21.15.50:0/2702078298 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1d64067eb0 con 0x7f1d640d1210 2024-04-02T16:21:54.132 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.132+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d640750e0 con 0x7f1d640d1210 2024-04-02T16:21:54.133 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.133+0000 7f1d5affd640 1 -- 172.21.15.50:0/2702078298 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f1d6007c190 con 0x7f1d640d1210 2024-04-02T16:21:54.133 INFO:teuthology.misc.health.smithi050.stdout: 2024-04-02T16:21:54.133 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":16,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:21:50.695620+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:21:54.135 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1d4806c8f0 msgr2=0x7f1d4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.135 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1d4806c8f0 0x7f1d4806edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d6415ec00 tx=0x7f1d54009040 comp rx=0 tx=0).stop 2024-04-02T16:21:54.135 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d640d1210 msgr2=0x7f1d640ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.135 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d640d1210 0x7f1d640ca710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1d60000c00 tx=0x7f1d60004730 comp rx=0 tx=0).stop 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 shutdown_connections 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d6415e460 0x7f1d64158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d640d3980 0x7f1d64157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1d4806c8f0 0x7f1d4806edb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 --2- 172.21.15.50:0/2702078298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d640d1210 0x7f1d640ca710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 >> 172.21.15.50:0/2702078298 conn(0x7f1d64072dd0 msgr2=0x7f1d640cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 shutdown_connections 2024-04-02T16:21:54.136 INFO:teuthology.misc.health.smithi050.stderr:2024-04-02T16:21:54.135+0000 7f1d6c6d6640 1 -- 172.21.15.50:0/2702078298 wait complete. 2024-04-02T16:21:54.144 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-02T16:21:54.145 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-02T16:21:54.145 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-02T16:21:54.145 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-02T16:21:54.147 INFO:tasks.ceph.mds.a:Started 2024-04-02T16:21:54.147 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-02T16:21:54.148 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-02T16:21:54.149 INFO:tasks.ceph.mds.c:Started 2024-04-02T16:21:54.149 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-02T16:21:54.150 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-02T16:21:54.152 INFO:tasks.ceph.mds.b:Started 2024-04-02T16:21:54.152 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-02T16:21:54.152 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-02T16:21:54.154 INFO:tasks.ceph.mds.d:Started 2024-04-02T16:21:54.154 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-02T16:21:54.155 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:21:54.224 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-04-02T16:21:54.224 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-04-02T16:21:54.239 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-02T16:21:54.239 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-02T16:21:54.240 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-02T16:21:54.239+0000 7f8089343640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-02T16:21:54.240 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-04-02T16:21:54.240+0000 7fb456c8e640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-04-02T16:21:54.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.260+0000 7f7f76160640 1 Processor -- start 2024-04-02T16:21:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.260+0000 7f7f76160640 1 -- start start 2024-04-02T16:21:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f76160640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f7015e570 0x7f7f700cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f76160640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f70162f70 0x7f7f70162340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f76160640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f700d05b0 con 0x7f7f700d0130 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f76160640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7f700d11c0 con 0x7f7f70162f70 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f76160640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f700d0890 con 0x7f7f7015e570 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f6ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f7f700d0130 0x7f7f700c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44314/0 (socket says 172.21.15.50:44314) 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f6ffff640 1 -- 172.21.15.50:0/446660126 learned_addr learned my addr 172.21.15.50:0/446660126 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f6effd640 1 --2- 172.21.15.50:0/446660126 >> v2:172.21.15.188:3300/0 conn(0x7f7f70162f70 0x7f7f70162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f6f7fe640 1 --2- 172.21.15.50:0/446660126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f7015e570 0x7f7f700cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.261+0000 7f7f6e7fc640 1 -- 172.21.15.50:0/446660126 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 830654817 0 0) 0x7f7f700d05b0 con 0x7f7f700d0130 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6e7fc640 1 -- 172.21.15.50:0/446660126 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f58003520 con 0x7f7f700d0130 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6e7fc640 1 -- 172.21.15.50:0/446660126 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165299178 0 0) 0x7f7f58003520 con 0x7f7f700d0130 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6f7fe640 1 -- 172.21.15.50:0/446660126 >> v2:172.21.15.188:3300/0 conn(0x7f7f70162f70 msgr2=0x7f7f70162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6f7fe640 1 --2- 172.21.15.50:0/446660126 >> v2:172.21.15.188:3300/0 conn(0x7f7f70162f70 0x7f7f70162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6f7fe640 1 -- 172.21.15.50:0/446660126 >> v1:172.21.15.50:6789/0 conn(0x7f7f700d0130 legacy=0x7f7f700c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6f7fe640 1 -- 172.21.15.50:0/446660126 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f700d0b70 con 0x7f7f7015e570 2024-04-02T16:21:54.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6effd640 1 --2- 172.21.15.50:0/446660126 >> v2:172.21.15.188:3300/0 conn(0x7f7f70162f70 0x7f7f70162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:54.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.262+0000 7f7f6f7fe640 1 --2- 172.21.15.50:0/446660126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f7015e570 0x7f7f700cfbf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f700d1340 tx=0x7f7f6002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71fa4f6a0929bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f6e7fc640 1 -- 172.21.15.50:0/446660126 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60037070 con 0x7f7f7015e570 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f6e7fc640 1 -- 172.21.15.50:0/446660126 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60030510 con 0x7f7f7015e570 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f6e7fc640 1 -- 172.21.15.50:0/446660126 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68001a10 con 0x7f7f7015e570 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 -- 172.21.15.50:0/446660126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f7015e570 msgr2=0x7f7f700cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 --2- 172.21.15.50:0/446660126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f7015e570 0x7f7f700cfbf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f700d1340 tx=0x7f7f6002b770 comp rx=0 tx=0).stop 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 -- 172.21.15.50:0/446660126 shutdown_connections 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 --2- 172.21.15.50:0/446660126 >> v2:172.21.15.188:3300/0 conn(0x7f7f70162f70 0x7f7f70162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 --2- 172.21.15.50:0/446660126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f7015e570 0x7f7f700cfbf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 -- 172.21.15.50:0/446660126 >> 172.21.15.50:0/446660126 conn(0x7f7f7009d5a0 msgr2=0x7f7f700d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 -- 172.21.15.50:0/446660126 shutdown_connections 2024-04-02T16:21:54.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.263+0000 7f7f76160640 1 -- 172.21.15.50:0/446660126 wait complete. 2024-04-02T16:21:54.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.264+0000 7f7f76160640 1 Processor -- start 2024-04-02T16:21:54.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f76160640 1 -- start start 2024-04-02T16:21:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f76160640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f700d0130 0x7f7f70159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f76160640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f7015e570 0x7f7f70142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f76160640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70162f70 0x7f7f70156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f76160640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700d17d0 con 0x7f7f700d0130 2024-04-02T16:21:54.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f76160640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700d1650 con 0x7f7f7015e570 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f76160640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f701591c0 con 0x7f7f70162f70 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f6f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f700d0130 0x7f7f70159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f6f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f700d0130 0x7f7f70159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55286/0 (socket says 172.21.15.50:55286) 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f6f7fe640 1 -- 172.21.15.50:0/1802661404 learned_addr learned my addr 172.21.15.50:0/1802661404 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f6effd640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f7015e570 0x7f7f70142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.265+0000 7f7f6ffff640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70162f70 0x7f7f70156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f6f7fe640 1 -- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70162f70 msgr2=0x7f7f70156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f6f7fe640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70162f70 0x7f7f70156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f6f7fe640 1 -- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f7015e570 msgr2=0x7f7f70142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f6f7fe640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f7015e570 0x7f7f70142160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f6f7fe640 1 -- 172.21.15.50:0/1802661404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f701585b0 con 0x7f7f700d0130 2024-04-02T16:21:54.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f6f7fe640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f700d0130 0x7f7f70159650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000c00 tx=0x7f7f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f57fff640 1 -- 172.21.15.50:0/1802661404 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60041020 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f57fff640 1 -- 172.21.15.50:0/1802661404 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6003c890 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f57fff640 1 -- 172.21.15.50:0/1802661404 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60037040 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f70158890 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.266+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f70159b90 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.267+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f700756f0 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.267+0000 7f7f57fff640 1 -- 172.21.15.50:0/1802661404 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f60044020 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.268+0000 7f7f57fff640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.268+0000 7f7f57fff640 1 -- 172.21.15.50:0/1802661404 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7f600ad110 con 0x7f7f700d0130 2024-04-02T16:21:54.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.268+0000 7f7f6effd640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.269+0000 7f7f6effd640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009560 tx=0x7f7f64009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.272+0000 7f7f57fff640 1 -- 172.21.15.50:0/1802661404 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7f6004b030 con 0x7f7f700d0130 2024-04-02T16:21:54.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.456+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f7006d780 con 0x7f7f700d0130 2024-04-02T16:21:54.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.457+0000 7f7f57fff640 1 -- 172.21.15.50:0/1802661404 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f7f6007b1b0 con 0x7f7f700d0130 2024-04-02T16:21:54.458 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:54.458 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":16,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:21:50.695620+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7f5c06c8f0 msgr2=0x7f7f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009560 tx=0x7f7f64009190 comp rx=0 tx=0).stop 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f700d0130 msgr2=0x7f7f70159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f700d0130 0x7f7f70159650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000c00 tx=0x7f7f600046c0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 shutdown_connections 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70162f70 0x7f7f70156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f7015e570 0x7f7f70142160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 --2- 172.21.15.50:0/1802661404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f700d0130 0x7f7f70159650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 >> 172.21.15.50:0/1802661404 conn(0x7f7f7009d5a0 msgr2=0x7f7f7015f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:54.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 shutdown_connections 2024-04-02T16:21:54.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.459+0000 7f7f76160640 1 -- 172.21.15.50:0/1802661404 wait complete. 2024-04-02T16:21:54.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:21:54.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:21:54.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.574+0000 7fd0bec3a640 1 Processor -- start 2024-04-02T16:21:54.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.574+0000 7fd0bec3a640 1 -- start start 2024-04-02T16:21:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.575+0000 7fd0bec3a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0b80b4740 0x7fd0b806f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.575+0000 7fd0bec3a640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b80d0960 con 0x7fd0b815c340 2024-04-02T16:21:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.575+0000 7fd0bec3a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b80d0c40 con 0x7fd0b815c720 2024-04-02T16:21:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.575+0000 7fd0bec3a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b80d0710 con 0x7fd0b80b4740 2024-04-02T16:21:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.575+0000 7fd0bc9d6640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd0b815c340 0x7fd0b80cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44322/0 (socket says 172.21.15.50:44322) 2024-04-02T16:21:54.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.575+0000 7fd0bc9d6640 1 -- 172.21.15.50:0/1517376024 learned_addr learned my addr 172.21.15.50:0/1517376024 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829626290 0 0) 0x7fd0b80d0960 con 0x7fd0b815c340 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0bd1d7640 1 --2- 172.21.15.50:0/1517376024 >> v2:172.21.15.188:3300/0 conn(0x7fd0b80b4740 0x7fd0b806f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0a0003520 con 0x7fd0b815c340 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1291436803 0 0) 0x7fd0a0003520 con 0x7fd0b815c340 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 >> v2:172.21.15.188:3300/0 conn(0x7fd0b80b4740 msgr2=0x7fd0b806f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 --2- 172.21.15.50:0/1517376024 >> v2:172.21.15.188:3300/0 conn(0x7fd0b80b4740 0x7fd0b806f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 >> v1:172.21.15.188:6789/0 conn(0x7fd0b815c720 legacy=0x7fd0b80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b80d0f20 con 0x7fd0b815c340 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fd0a8001670 con 0x7fd0b815c340 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 >> v1:172.21.15.50:6789/0 conn(0x7fd0b815c340 legacy=0x7fd0b80cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00039b0 0x7fd0a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0a0004800 0x7fd0a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00051c0 0x7fd0a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b80d0710 con 0x7fd0a0004800 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a000c060 con 0x7fd0a00051c0 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a000c260 con 0x7fd0a00039b0 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b7fff640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0a0004800 0x7fd0a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0bc9d6640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00039b0 0x7fd0a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0bd1d7640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00051c0 0x7fd0a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b7fff640 1 -- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00039b0 msgr2=0x7fd0a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b7fff640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00039b0 0x7fd0a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b7fff640 1 -- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00051c0 msgr2=0x7fd0a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:54.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b7fff640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00051c0 0x7fd0a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0b7fff640 1 -- 172.21.15.50:0/1517376024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a0003520 con 0x7fd0a0004800 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.576+0000 7fd0bd1d7640 1 -- 172.21.15.50:0/1517376024 reap_dead start 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.577+0000 7fd0b7fff640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0a0004800 0x7fd0a0004c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac001750 tx=0x7fd0a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5655fa6273fc2721 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.577+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac001370 con 0x7fd0a0004800 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.577+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ac0052d0 con 0x7fd0a0004800 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.577+0000 7fd0b77fe640 1 -- 172.21.15.50:0/1517376024 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac003830 con 0x7fd0a0004800 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.578+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1517376024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0a0004800 msgr2=0x7fd0a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.578+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0a0004800 0x7fd0a0004c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac001750 tx=0x7fd0a0005830 comp rx=0 tx=0).stop 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.578+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1517376024 shutdown_connections 2024-04-02T16:21:54.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.578+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1517376024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0a0004800 0x7fd0a0004c80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.578+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1517376024 >> 172.21.15.50:0/1517376024 conn(0x7fd0b80b3c20 msgr2=0x7fd0b80b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:54.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.578+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1517376024 shutdown_connections 2024-04-02T16:21:54.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.578+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1517376024 wait complete. 2024-04-02T16:21:54.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.579+0000 7fd0bec3a640 1 Processor -- start 2024-04-02T16:21:54.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.579+0000 7fd0bec3a640 1 -- start start 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bec3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a0004800 0x7fd0b806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bec3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b80b73f0 0x7fd0b806de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bec3a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b806e3d0 0x7fd0b80afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bec3a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b806f630 con 0x7fd0b806e3d0 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bec3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b80cd030 con 0x7fd0a0004800 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bec3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b806f7b0 con 0x7fd0b80b73f0 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b806e3d0 0x7fd0b80afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b806e3d0 0x7fd0b80afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55306/0 (socket says 172.21.15.50:55306) 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 -- 172.21.15.50:0/1990854696 learned_addr learned my addr 172.21.15.50:0/1990854696 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0b7fff640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b80b73f0 0x7fd0b806de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bc9d6640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a0004800 0x7fd0b806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 -- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b80b73f0 msgr2=0x7fd0b806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b80b73f0 0x7fd0b806de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 -- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a0004800 msgr2=0x7fd0b806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a0004800 0x7fd0b806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 -- 172.21.15.50:0/1990854696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b806cb70 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bc9d6640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a0004800 0x7fd0b806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0b7fff640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b80b73f0 0x7fd0b806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.580+0000 7fd0bd1d7640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b806e3d0 0x7fd0b80afec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80d31b0 tx=0x7fd0b000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.581+0000 7fd0b57fa640 1 -- 172.21.15.50:0/1990854696 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b000fce0 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.581+0000 7fd0b57fa640 1 -- 172.21.15.50:0/1990854696 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b001d070 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.581+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b806cdf0 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.581+0000 7fd0b57fa640 1 -- 172.21.15.50:0/1990854696 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0015670 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.581+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b806d3b0 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.582+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b8075760 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.582+0000 7fd0b57fa640 1 -- 172.21.15.50:0/1990854696 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0b0015810 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.583+0000 7fd0b57fa640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd0a406c8f0 0x7fd0a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.583+0000 7fd0b57fa640 1 -- 172.21.15.50:0/1990854696 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd0b008c1d0 con 0x7fd0b806e3d0 2024-04-02T16:21:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.583+0000 7fd0bc9d6640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd0a406c8f0 0x7fd0a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.583+0000 7fd0bc9d6640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd0a406c8f0 0x7fd0a406edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8001eb0 tx=0x7fd0a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.587+0000 7fd0b57fa640 1 -- 172.21.15.50:0/1990854696 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd0b806cb70 con 0x7fd0b806e3d0 2024-04-02T16:21:54.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.770+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0b80b80b0 con 0x7fd0b806e3d0 2024-04-02T16:21:54.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.771+0000 7fd0b57fa640 1 -- 172.21.15.50:0/1990854696 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fd0b0010040 con 0x7fd0b806e3d0 2024-04-02T16:21:54.771 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:21:54.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd0a406c8f0 msgr2=0x7fd0a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd0a406c8f0 0x7fd0a406edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8001eb0 tx=0x7fd0a8009040 comp rx=0 tx=0).stop 2024-04-02T16:21:54.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b806e3d0 msgr2=0x7fd0b80afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b806e3d0 0x7fd0b80afec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80d31b0 tx=0x7fd0b000f1b0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 shutdown_connections 2024-04-02T16:21:54.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd0a406c8f0 0x7fd0a406edb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b806e3d0 0x7fd0b80afec0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b80b73f0 0x7fd0b806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 --2- 172.21.15.50:0/1990854696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a0004800 0x7fd0b806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 >> 172.21.15.50:0/1990854696 conn(0x7fd0b80b3c20 msgr2=0x7fd0b80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:54.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 shutdown_connections 2024-04-02T16:21:54.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.773+0000 7fd0bec3a640 1 -- 172.21.15.50:0/1990854696 wait complete. 2024-04-02T16:21:54.784 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:21:54.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.885+0000 7fba04b0e640 1 Processor -- start 2024-04-02T16:21:54.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.885+0000 7fba04b0e640 1 -- start start 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fba04b0e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba0015c330 0x7fba000cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fba04b0e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba000d0da0 0x7fba000cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fba04b0e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba000ccba0 con 0x7fba000b46c0 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fba04b0e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fba000d01c0 con 0x7fba0015c330 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fba04b0e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fba000cd770 con 0x7fba000d0da0 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fb9fed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba000d0da0 0x7fba000cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fb9fe59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba0015c330 0x7fba000cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fb9fdd9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fba000b46c0 0x7fba000d0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:37076/0 (socket says 172.21.15.50:37076) 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.886+0000 7fb9fdd9b640 1 -- 172.21.15.50:0/2967917490 learned_addr learned my addr 172.21.15.50:0/2967917490 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.887+0000 7fb9fe59c640 1 -- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba000d0da0 msgr2=0x7fba000cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.887+0000 7fb9fe59c640 1 --2- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba000d0da0 0x7fba000cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.887+0000 7fb9fe59c640 1 -- 172.21.15.50:0/2967917490 >> v1:172.21.15.188:6789/0 conn(0x7fba000b46c0 legacy=0x7fba000d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.887+0000 7fb9fe59c640 1 -- 172.21.15.50:0/2967917490 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba000cce40 con 0x7fba0015c330 2024-04-02T16:21:54.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.887+0000 7fb9fed9d640 1 --2- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba000d0da0 0x7fba000cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fb9fe59c640 1 --2- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba0015c330 0x7fba000cdc00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009d90 tx=0x7fb9f002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=792343500fa529a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fb9fd59a640 1 -- 172.21.15.50:0/2967917490 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0039070 con 0x7fba0015c330 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fb9fd59a640 1 -- 172.21.15.50:0/2967917490 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f002cc50 con 0x7fba0015c330 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fb9fd59a640 1 -- 172.21.15.50:0/2967917490 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0034440 con 0x7fba0015c330 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fba04b0e640 1 -- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba0015c330 msgr2=0x7fba000cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fba04b0e640 1 --2- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba0015c330 0x7fba000cdc00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009d90 tx=0x7fb9f002bea0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fba04b0e640 1 -- 172.21.15.50:0/2967917490 shutdown_connections 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fba04b0e640 1 --2- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba000d0da0 0x7fba000cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fba04b0e640 1 --2- 172.21.15.50:0/2967917490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba0015c330 0x7fba000cdc00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.888+0000 7fba04b0e640 1 -- 172.21.15.50:0/2967917490 >> 172.21.15.50:0/2967917490 conn(0x7fba000b3ba0 msgr2=0x7fba000b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.889+0000 7fba04b0e640 1 -- 172.21.15.50:0/2967917490 shutdown_connections 2024-04-02T16:21:54.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.889+0000 7fba04b0e640 1 -- 172.21.15.50:0/2967917490 wait complete. 2024-04-02T16:21:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.890+0000 7fba04b0e640 1 Processor -- start 2024-04-02T16:21:54.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.890+0000 7fba04b0e640 1 -- start start 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba000b46c0 0x7fba0009f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba000d0da0 0x7fba000aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba0015c330 0x7fba000cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba0006f610 con 0x7fba000b46c0 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba000cd400 con 0x7fba000d0da0 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fba0006f790 con 0x7fba0015c330 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba000b46c0 0x7fba0009f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba000b46c0 0x7fba0009f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55316/0 (socket says 172.21.15.50:55316) 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 -- 172.21.15.50:0/27720453 learned_addr learned my addr 172.21.15.50:0/27720453 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fed9d640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba0015c330 0x7fba000cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fdd9b640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba000d0da0 0x7fba000aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 -- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba0015c330 msgr2=0x7fba000cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba0015c330 0x7fba000cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 -- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba000d0da0 msgr2=0x7fba000aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba000d0da0 0x7fba000aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 -- 172.21.15.50:0/27720453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0009cfd0 con 0x7fba000b46c0 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9fe59c640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba000b46c0 0x7fba0009f6e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009d90 tx=0x7fb9f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9f6ffd640 1 -- 172.21.15.50:0/27720453 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0039050 con 0x7fba000b46c0 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9f6ffd640 1 -- 172.21.15.50:0/27720453 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f0045070 con 0x7fba000b46c0 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fb9f6ffd640 1 -- 172.21.15.50:0/27720453 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f003e610 con 0x7fba000b46c0 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0009d2b0 con 0x7fba000b46c0 2024-04-02T16:21:54.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.891+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001f5b30 con 0x7fba000b46c0 2024-04-02T16:21:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.892+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba000b4030 con 0x7fba000b46c0 2024-04-02T16:21:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.893+0000 7fb9f6ffd640 1 -- 172.21.15.50:0/27720453 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9f003e7b0 con 0x7fba000b46c0 2024-04-02T16:21:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.893+0000 7fb9f6ffd640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9e006c8f0 0x7fb9e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.893+0000 7fb9f6ffd640 1 -- 172.21.15.50:0/27720453 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb9f00ae130 con 0x7fba000b46c0 2024-04-02T16:21:54.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.893+0000 7fb9fdd9b640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9e006c8f0 0x7fb9e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:54.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.894+0000 7fb9fdd9b640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9e006c8f0 0x7fb9e006edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fba001f6c30 tx=0x7fb9e8007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:54.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:54.897+0000 7fb9f6ffd640 1 -- 172.21.15.50:0/27720453 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fba000b4030 con 0x7fba000b46c0 2024-04-02T16:21:55.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.121+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba000756f0 con 0x7fba000b46c0 2024-04-02T16:21:55.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.121+0000 7fb9f6ffd640 1 -- 172.21.15.50:0/27720453 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fb9f007c2f0 con 0x7fba000b46c0 2024-04-02T16:21:55.121 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:55.122 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T16:21:55.122 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 2 2024-04-02T16:21:55.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9e006c8f0 msgr2=0x7fb9e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9e006c8f0 0x7fb9e006edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fba001f6c30 tx=0x7fb9e8007e80 comp rx=0 tx=0).stop 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba000b46c0 msgr2=0x7fba0009f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba000b46c0 0x7fba0009f6e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009d90 tx=0x7fb9f0004a40 comp rx=0 tx=0).stop 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 shutdown_connections 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba0015c330 0x7fba000cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba000d0da0 0x7fba000aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9e006c8f0 0x7fb9e006edb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 --2- 172.21.15.50:0/27720453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba000b46c0 0x7fba0009f6e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 >> 172.21.15.50:0/27720453 conn(0x7fba000b3ba0 msgr2=0x7fba000b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 shutdown_connections 2024-04-02T16:21:55.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:55.123+0000 7fba04b0e640 1 -- 172.21.15.50:0/27720453 wait complete. 2024-04-02T16:21:57.136 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:21:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.249+0000 7f5bfb300640 1 Processor -- start 2024-04-02T16:21:57.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.250+0000 7f5bfb300640 1 -- start start 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.250+0000 7f5bfb300640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5bf4162f70 0x7f5bf4162340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.250+0000 7f5bfb300640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf40d0300 con 0x7f5bf40d1650 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.250+0000 7f5bfb300640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf40d05b0 con 0x7f5bf415e570 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.250+0000 7f5bfb300640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf40d11c0 con 0x7f5bf4162f70 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.251+0000 7f5bf889b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5bf4162f70 0x7f5bf4162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.251+0000 7f5bf989d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5bf40d1650 0x7f5bf40c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42318/0 (socket says 172.21.15.50:42318) 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.251+0000 7f5bf989d640 1 -- 172.21.15.50:0/1738793052 learned_addr learned my addr 172.21.15.50:0/1738793052 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:57.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.251+0000 7f5bf889b640 1 -- 172.21.15.50:0/1738793052 >> v1:172.21.15.188:6789/0 conn(0x7f5bf415e570 legacy=0x7f5bf40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:57.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.251+0000 7f5bf889b640 1 -- 172.21.15.50:0/1738793052 >> v1:172.21.15.50:6789/0 conn(0x7f5bf40d1650 legacy=0x7f5bf40c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.251+0000 7f5bf889b640 1 -- 172.21.15.50:0/1738793052 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf40d0890 con 0x7f5bf4162f70 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bf889b640 1 --2- 172.21.15.50:0/1738793052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf4162f70 0x7f5bf4162340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5be40097a0 tx=0x7f5be402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802b1cf79c4b550e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bebfff640 1 -- 172.21.15.50:0/1738793052 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4035020 con 0x7f5bf4162f70 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bebfff640 1 -- 172.21.15.50:0/1738793052 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4031830 con 0x7f5bf4162f70 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bebfff640 1 -- 172.21.15.50:0/1738793052 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4031c00 con 0x7f5bf4162f70 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bfb300640 1 -- 172.21.15.50:0/1738793052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf4162f70 msgr2=0x7f5bf4162340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1738793052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf4162f70 0x7f5bf4162340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5be40097a0 tx=0x7f5be402b8d0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bfb300640 1 -- 172.21.15.50:0/1738793052 shutdown_connections 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1738793052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf4162f70 0x7f5bf4162340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bfb300640 1 -- 172.21.15.50:0/1738793052 >> 172.21.15.50:0/1738793052 conn(0x7f5bf409d5a0 msgr2=0x7f5bf40d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:57.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bfb300640 1 -- 172.21.15.50:0/1738793052 shutdown_connections 2024-04-02T16:21:57.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.252+0000 7f5bfb300640 1 -- 172.21.15.50:0/1738793052 wait complete. 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 Processor -- start 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 -- start start 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bf40d1650 0x7f5bf40b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf415e570 0x7f5bf4158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bf4162f70 0x7f5bf40b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf415e950 con 0x7f5bf415e570 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40d0e50 con 0x7f5bf40d1650 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bfb300640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf415ead0 con 0x7f5bf4162f70 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bf889b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf415e570 0x7f5bf4158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.254+0000 7f5bf889b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf415e570 0x7f5bf4158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50364/0 (socket says 172.21.15.50:50364) 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf889b640 1 -- 172.21.15.50:0/1178777140 learned_addr learned my addr 172.21.15.50:0/1178777140 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:57.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf989d640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bf4162f70 0x7f5bf40b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf909c640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bf40d1650 0x7f5bf40b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf889b640 1 -- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bf4162f70 msgr2=0x7f5bf40b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf889b640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bf4162f70 0x7f5bf40b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf889b640 1 -- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bf40d1650 msgr2=0x7f5bf40b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf889b640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bf40d1650 0x7f5bf40b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf889b640 1 -- 172.21.15.50:0/1178777140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf40b71e0 con 0x7f5bf415e570 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf909c640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bf40d1650 0x7f5bf40b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bf889b640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf415e570 0x7f5bf4158a50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098c0 tx=0x7f5be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5be9ffb640 1 -- 172.21.15.50:0/1178777140 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4040070 con 0x7f5bf415e570 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5be9ffb640 1 -- 172.21.15.50:0/1178777140 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4039a00 con 0x7f5bf415e570 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf40b74c0 con 0x7f5bf415e570 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5be9ffb640 1 -- 172.21.15.50:0/1178777140 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4035040 con 0x7f5bf415e570 2024-04-02T16:21:57.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.255+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf4158250 con 0x7f5bf415e570 2024-04-02T16:21:57.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.256+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf40756f0 con 0x7f5bf415e570 2024-04-02T16:21:57.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.257+0000 7f5be9ffb640 1 -- 172.21.15.50:0/1178777140 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5be4038020 con 0x7f5bf415e570 2024-04-02T16:21:57.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.257+0000 7f5be9ffb640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5bd806c8f0 0x7f5bd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.257+0000 7f5be9ffb640 1 -- 172.21.15.50:0/1178777140 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5be40ad2d0 con 0x7f5bf415e570 2024-04-02T16:21:57.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.257+0000 7f5bf909c640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5bd806c8f0 0x7f5bd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.258+0000 7f5bf909c640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5bd806c8f0 0x7f5bd806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5bf41f7f20 tx=0x7f5bf0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.262+0000 7f5be9ffb640 1 -- 172.21.15.50:0/1178777140 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5be407ba50 con 0x7f5bf415e570 2024-04-02T16:21:57.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.485+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bf40d1ae0 con 0x7f5bf415e570 2024-04-02T16:21:57.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.486+0000 7f5be9ffb640 1 -- 172.21.15.50:0/1178777140 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7f5be407b350 con 0x7f5bf415e570 2024-04-02T16:21:57.486 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:57.486 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T16:21:57.487 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 3 2024-04-02T16:21:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5bd806c8f0 msgr2=0x7f5bd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5bd806c8f0 0x7f5bd806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5bf41f7f20 tx=0x7f5bf0009040 comp rx=0 tx=0).stop 2024-04-02T16:21:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf415e570 msgr2=0x7f5bf4158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf415e570 0x7f5bf4158a50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098c0 tx=0x7f5be40046c0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 shutdown_connections 2024-04-02T16:21:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bf4162f70 0x7f5bf40b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5bd806c8f0 0x7f5bd806edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bf415e570 0x7f5bf4158a50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 --2- 172.21.15.50:0/1178777140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bf40d1650 0x7f5bf40b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 >> 172.21.15.50:0/1178777140 conn(0x7f5bf409d5a0 msgr2=0x7f5bf415fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 shutdown_connections 2024-04-02T16:21:57.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.488+0000 7f5bfb300640 1 -- 172.21.15.50:0/1178777140 wait complete. 2024-04-02T16:21:57.500 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:21:57.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.601+0000 7f8fc791c640 1 Processor -- start 2024-04-02T16:21:57.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.601+0000 7f8fc791c640 1 -- start start 2024-04-02T16:21:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc791c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8fc015e510 0x7f8fc00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc791c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8fc00d15a0 0x7f8fc00d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc791c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc00ceda0 con 0x7f8fc00d2800 2024-04-02T16:21:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc791c640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc00d19a0 con 0x7f8fc015e510 2024-04-02T16:21:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc791c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc00d31c0 con 0x7f8fc00d15a0 2024-04-02T16:21:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc56b8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8fc015e510 0x7f8fc00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc56b8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8fc015e510 0x7f8fc00cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50366/0 (socket says 172.21.15.50:50366) 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc56b8640 1 -- 172.21.15.50:0/393599159 learned_addr learned my addr 172.21.15.50:0/393599159 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc5eb9640 1 --2- 172.21.15.50:0/393599159 >> v2:172.21.15.188:3300/0 conn(0x7f8fc00d15a0 0x7f8fc00d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc56b8640 1 -- 172.21.15.50:0/393599159 >> v2:172.21.15.188:3300/0 conn(0x7f8fc00d15a0 msgr2=0x7f8fc00d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc56b8640 1 --2- 172.21.15.50:0/393599159 >> v2:172.21.15.188:3300/0 conn(0x7f8fc00d15a0 0x7f8fc00d2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc56b8640 1 -- 172.21.15.50:0/393599159 >> v1:172.21.15.188:6789/0 conn(0x7f8fc00d2800 legacy=0x7f8fc00d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.602+0000 7f8fc56b8640 1 -- 172.21.15.50:0/393599159 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc00cf040 con 0x7f8fc015e510 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fc56b8640 1 --2- 172.21.15.50:0/393599159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cfe00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fc008e0f0 tx=0x7f8fbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af58ba5016ff8c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fb7fff640 1 -- 172.21.15.50:0/393599159 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0315a0 con 0x7f8fc015e510 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fb7fff640 1 -- 172.21.15.50:0/393599159 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc037070 con 0x7f8fc015e510 2024-04-02T16:21:57.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fb7fff640 1 -- 172.21.15.50:0/393599159 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc031960 con 0x7f8fc015e510 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fc791c640 1 -- 172.21.15.50:0/393599159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 msgr2=0x7f8fc00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fc791c640 1 --2- 172.21.15.50:0/393599159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cfe00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fc008e0f0 tx=0x7f8fbc02b840 comp rx=0 tx=0).stop 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fc791c640 1 -- 172.21.15.50:0/393599159 shutdown_connections 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fc791c640 1 --2- 172.21.15.50:0/393599159 >> v2:172.21.15.188:3300/0 conn(0x7f8fc00d15a0 0x7f8fc00d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fc791c640 1 --2- 172.21.15.50:0/393599159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cfe00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.603+0000 7f8fc791c640 1 -- 172.21.15.50:0/393599159 >> 172.21.15.50:0/393599159 conn(0x7f8fc009d570 msgr2=0x7f8fc00d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.604+0000 7f8fc791c640 1 -- 172.21.15.50:0/393599159 shutdown_connections 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.604+0000 7f8fc791c640 1 -- 172.21.15.50:0/393599159 wait complete. 2024-04-02T16:21:57.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 Processor -- start 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 -- start start 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00d15a0 0x7f8fc00cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00d2800 0x7f8fc00cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc015e8f0 con 0x7f8fc015e510 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc00cf600 con 0x7f8fc00d15a0 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc791c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc00b3ad0 con 0x7f8fc00d2800 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.605+0000 7f8fc5eb9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50380/0 (socket says 172.21.15.50:50380) 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 -- 172.21.15.50:0/1709044562 learned_addr learned my addr 172.21.15.50:0/1709044562 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc56b8640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00d15a0 0x7f8fc00cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc4eb7640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00d2800 0x7f8fc00cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 -- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00d2800 msgr2=0x7f8fc00cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00d2800 0x7f8fc00cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 -- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00d15a0 msgr2=0x7f8fc00cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00d15a0 0x7f8fc00cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 -- 172.21.15.50:0/1709044562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc015f350 con 0x7f8fc015e510 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc5eb9640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cb8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800cd70 tx=0x7f8fb800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fb5ffb640 1 -- 172.21.15.50:0/1709044562 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8003b90 con 0x7f8fc015e510 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fb5ffb640 1 -- 172.21.15.50:0/1709044562 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb8003d30 con 0x7f8fc015e510 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc015f630 con 0x7f8fc015e510 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fb5ffb640 1 -- 172.21.15.50:0/1709044562 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8004240 con 0x7f8fc015e510 2024-04-02T16:21:57.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.606+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc015fbf0 con 0x7f8fc015e510 2024-04-02T16:21:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.607+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc00756f0 con 0x7f8fc015e510 2024-04-02T16:21:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.608+0000 7f8fb5ffb640 1 -- 172.21.15.50:0/1709044562 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fb80043e0 con 0x7f8fc015e510 2024-04-02T16:21:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.608+0000 7f8fb5ffb640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8fa406c8f0 0x7f8fa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.608+0000 7f8fb5ffb640 1 -- 172.21.15.50:0/1709044562 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f8fb808c1b0 con 0x7f8fc015e510 2024-04-02T16:21:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.608+0000 7f8fc56b8640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8fa406c8f0 0x7f8fa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.609+0000 7f8fc56b8640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8fa406c8f0 0x7f8fa406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc036000 tx=0x7f8fbc03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.612+0000 7f8fb5ffb640 1 -- 172.21.15.50:0/1709044562 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8fb805a2e0 con 0x7f8fc015e510 2024-04-02T16:21:57.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.802+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fc00b4730 con 0x7f8fc015e510 2024-04-02T16:21:57.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.803+0000 7f8fb5ffb640 1 -- 172.21.15.50:0/1709044562 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f8fb805e5e0 con 0x7f8fc015e510 2024-04-02T16:21:57.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:57.803 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":16,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:21:50.695620+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:21:57.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8fa406c8f0 msgr2=0x7f8fa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8fa406c8f0 0x7f8fa406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc036000 tx=0x7f8fbc03d040 comp rx=0 tx=0).stop 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 msgr2=0x7f8fc00cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cb8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800cd70 tx=0x7f8fb800a660 comp rx=0 tx=0).stop 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 shutdown_connections 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8fa406c8f0 0x7f8fa406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fc015e510 0x7f8fc00cb8a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00d2800 0x7f8fc00cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 --2- 172.21.15.50:0/1709044562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00d15a0 0x7f8fc00cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 >> 172.21.15.50:0/1709044562 conn(0x7f8fc009d570 msgr2=0x7f8fc00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 shutdown_connections 2024-04-02T16:21:57.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.805+0000 7f8fc791c640 1 -- 172.21.15.50:0/1709044562 wait complete. 2024-04-02T16:21:57.816 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:21:57.816 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:21:57.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.920+0000 7f9fec3a8640 1 Processor -- start 2024-04-02T16:21:57.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.920+0000 7f9fec3a8640 1 -- start start 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fec3a8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9fe4161c80 0x7f9fe40d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fec3a8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9fe40d2e50 0x7f9fe40d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fec3a8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe40d1200 0x7f9fe40cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fec3a8640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe40b10e0 con 0x7f9fe4161c80 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fec3a8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe40b1380 con 0x7f9fe40d2e50 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fec3a8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe40b1660 con 0x7f9fe40d1200 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9fe4161c80 0x7f9fe40d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9fe4161c80 0x7f9fe40d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50396/0 (socket says 172.21.15.50:50396) 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 -- 172.21.15.50:0/1147438280 learned_addr learned my addr 172.21.15.50:0/1147438280 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 -- 172.21.15.50:0/1147438280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe40d1200 msgr2=0x7f9fe40cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 --2- 172.21.15.50:0/1147438280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe40d1200 0x7f9fe40cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 -- 172.21.15.50:0/1147438280 >> v2:172.21.15.188:3300/0 conn(0x7f9fe40d2e50 msgr2=0x7f9fe40d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:57.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fe9943640 1 --2- 172.21.15.50:0/1147438280 >> v2:172.21.15.188:3300/0 conn(0x7f9fe40d2e50 0x7f9fe40d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 --2- 172.21.15.50:0/1147438280 >> v2:172.21.15.188:3300/0 conn(0x7f9fe40d2e50 0x7f9fe40d0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.921+0000 7f9fea144640 1 -- 172.21.15.50:0/1147438280 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40d3340 con 0x7f9fe4161c80 2024-04-02T16:21:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.922+0000 7f9fea144640 1 --2- 172.21.15.50:0/1147438280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe4161c80 0x7f9fe40d2910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9fe408cc30 tx=0x7f9fdc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a928d4c6cabbcfe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.922+0000 7f9fe9142640 1 -- 172.21.15.50:0/1147438280 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc02e070 con 0x7f9fe4161c80 2024-04-02T16:21:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.922+0000 7f9fe9142640 1 -- 172.21.15.50:0/1147438280 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc037070 con 0x7f9fe4161c80 2024-04-02T16:21:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.922+0000 7f9fe9142640 1 -- 172.21.15.50:0/1147438280 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc031a00 con 0x7f9fe4161c80 2024-04-02T16:21:57.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1147438280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe4161c80 msgr2=0x7f9fe40d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1147438280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe4161c80 0x7f9fe40d2910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9fe408cc30 tx=0x7f9fdc02b880 comp rx=0 tx=0).stop 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1147438280 shutdown_connections 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1147438280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe40d1200 0x7f9fe40cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1147438280 >> v2:172.21.15.188:3300/0 conn(0x7f9fe40d2e50 0x7f9fe40d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1147438280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe4161c80 0x7f9fe40d2910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1147438280 >> 172.21.15.50:0/1147438280 conn(0x7f9fe409d5e0 msgr2=0x7f9fe40d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1147438280 shutdown_connections 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.923+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1147438280 wait complete. 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.924+0000 7f9fec3a8640 1 Processor -- start 2024-04-02T16:21:57.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.924+0000 7f9fec3a8640 1 -- start start 2024-04-02T16:21:57.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fec3a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fe40d1200 0x7f9fe40c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fec3a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe40d2e50 0x7f9fe406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fec3a8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe4161c80 0x7f9fe40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fec3a8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40cf460 con 0x7f9fe40d2e50 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fec3a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40d0390 con 0x7f9fe40d1200 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fec3a8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe40cf5e0 con 0x7f9fe4161c80 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe40d2e50 0x7f9fe406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe40d2e50 0x7f9fe406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50402/0 (socket says 172.21.15.50:50402) 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 -- 172.21.15.50:0/1874183966 learned_addr learned my addr 172.21.15.50:0/1874183966 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fea144640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fe40d1200 0x7f9fe40c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fea945640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe4161c80 0x7f9fe40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 -- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe4161c80 msgr2=0x7f9fe40b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe4161c80 0x7f9fe40b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 -- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fe40d1200 msgr2=0x7f9fe40c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fe40d1200 0x7f9fe40c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 -- 172.21.15.50:0/1874183966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40cc760 con 0x7f9fe40d2e50 2024-04-02T16:21:57.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fea945640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe4161c80 0x7f9fe40b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.925+0000 7f9fe9943640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe40d2e50 0x7f9fe406d580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c7f0 tx=0x7f9fd800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.926+0000 7f9fe2ffd640 1 -- 172.21.15.50:0/1874183966 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8015070 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.926+0000 7f9fe2ffd640 1 -- 172.21.15.50:0/1874183966 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd8004220 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.926+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe40cca40 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.926+0000 7f9fe2ffd640 1 -- 172.21.15.50:0/1874183966 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8007620 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.926+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe4158da0 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.927+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe4075760 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.927+0000 7f9fe2ffd640 1 -- 172.21.15.50:0/1874183966 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fd80079d0 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.927+0000 7f9fe2ffd640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9fd006c920 0x7f9fd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.927+0000 7f9fe2ffd640 1 -- 172.21.15.50:0/1874183966 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9fd808b8a0 con 0x7f9fe40d2e50 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.928+0000 7f9fea144640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9fd006c920 0x7f9fd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.928+0000 7f9fea144640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9fd006c920 0x7f9fd006ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc036000 tx=0x7f9fdc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:57.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:57.932+0000 7f9fe2ffd640 1 -- 172.21.15.50:0/1874183966 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9fd805a020 con 0x7f9fe40d2e50 2024-04-02T16:21:58.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.113+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9fe40b9120 con 0x7f9fe40d2e50 2024-04-02T16:21:58.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.113+0000 7f9fe2ffd640 1 -- 172.21.15.50:0/1874183966 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f9fd8059920 con 0x7f9fe40d2e50 2024-04-02T16:21:58.114 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9fd006c920 msgr2=0x7f9fd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9fd006c920 0x7f9fd006ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc036000 tx=0x7f9fdc008030 comp rx=0 tx=0).stop 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe40d2e50 msgr2=0x7f9fe406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe40d2e50 0x7f9fe406d580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c7f0 tx=0x7f9fd800ccb0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 shutdown_connections 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fe4161c80 0x7f9fe40b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9fd006c920 0x7f9fd006ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9fe40d2e50 0x7f9fe406d580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 --2- 172.21.15.50:0/1874183966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fe40d1200 0x7f9fe40c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 >> 172.21.15.50:0/1874183966 conn(0x7f9fe409d5e0 msgr2=0x7f9fe40b9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:58.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 shutdown_connections 2024-04-02T16:21:58.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.115+0000 7f9fec3a8640 1 -- 172.21.15.50:0/1874183966 wait complete. 2024-04-02T16:21:58.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T16:21:58.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.228+0000 7f7167486640 1 Processor -- start 2024-04-02T16:21:58.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.228+0000 7f7167486640 1 -- start start 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7167486640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7160145680 0x7f71600cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7167486640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 0x7f7160072c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7167486640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f716006f900 con 0x7f716006b520 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7167486640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f71600cffb0 con 0x7f7160145680 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7167486640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f71600d0d80 con 0x7f7160072790 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7165222640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f716006b520 0x7f716006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42330/0 (socket says 172.21.15.50:42330) 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7165222640 1 -- 172.21.15.50:0/2288687371 learned_addr learned my addr 172.21.15.50:0/2288687371 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7165a23640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 0x7f7160072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7164a21640 1 --2- 172.21.15.50:0/2288687371 >> v2:172.21.15.188:3300/0 conn(0x7f7160145680 0x7f71600cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317067395 0 0) 0x7f715c001110 con 0x7f716006b520 2024-04-02T16:21:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f714c003520 con 0x7f716006b520 2024-04-02T16:21:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2381426048 0 0) 0x7f714c003520 con 0x7f716006b520 2024-04-02T16:21:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 msgr2=0x7f7160072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 0x7f7160072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 >> v2:172.21.15.188:3300/0 conn(0x7f7160145680 msgr2=0x7f71600cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:58.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 --2- 172.21.15.50:0/2288687371 >> v2:172.21.15.188:3300/0 conn(0x7f7160145680 0x7f71600cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7160145e70 con 0x7f716006b520 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f715c001840 con 0x7f716006b520 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 >> v1:172.21.15.50:6789/0 conn(0x7f716006b520 legacy=0x7f716006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f714c0039b0 0x7f714c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.229+0000 7f7157fff640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 0x7f714c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7157fff640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f714c004d40 0x7f714c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71600cffb0 con 0x7f714c004d40 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71600d0d80 con 0x7f714c0039b0 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f714c00b9c0 con 0x7f7160072790 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7165a23640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f714c004d40 0x7f714c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7165222640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f714c0039b0 0x7f714c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7164a21640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 0x7f714c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7165a23640 1 -- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 msgr2=0x7f714c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:58.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7165a23640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7160072790 0x7f714c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7165a23640 1 -- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f714c0039b0 msgr2=0x7f714c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7165a23640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f714c0039b0 0x7f714c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.230+0000 7f7165a23640 1 -- 172.21.15.50:0/2288687371 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c003520 con 0x7f714c004d40 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7165a23640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f714c004d40 0x7f714c0051c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7158009fc0 tx=0x7f714c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d0feed0ce050db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158039070 con 0x7f714c004d40 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715802cc20 con 0x7f714c004d40 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7157fff640 1 -- 172.21.15.50:0/2288687371 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158009c10 con 0x7f714c004d40 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7167486640 1 -- 172.21.15.50:0/2288687371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f714c004d40 msgr2=0x7f714c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7167486640 1 --2- 172.21.15.50:0/2288687371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f714c004d40 0x7f714c0051c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7158009fc0 tx=0x7f714c005830 comp rx=0 tx=0).stop 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7165a23640 1 -- 172.21.15.50:0/2288687371 reap_dead start 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7167486640 1 -- 172.21.15.50:0/2288687371 shutdown_connections 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7167486640 1 -- 172.21.15.50:0/2288687371 >> 172.21.15.50:0/2288687371 conn(0x7f71600b2d20 msgr2=0x7f71600b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7167486640 1 -- 172.21.15.50:0/2288687371 shutdown_connections 2024-04-02T16:21:58.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.231+0000 7f7167486640 1 -- 172.21.15.50:0/2288687371 wait complete. 2024-04-02T16:21:58.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.232+0000 7f7167486640 1 Processor -- start 2024-04-02T16:21:58.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7167486640 1 -- start start 2024-04-02T16:21:58.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7167486640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f716013eb90 0x7f71601f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7167486640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71600c8370 0x7f716006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7167486640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71600c68a0 0x7f716006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7167486640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71600d07c0 con 0x7f71600c68a0 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7167486640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71600d04e0 con 0x7f71600c8370 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7167486640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f71600d0aa0 con 0x7f716013eb90 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7165a23640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71600c68a0 0x7f716006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7165a23640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71600c68a0 0x7f716006e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50414/0 (socket says 172.21.15.50:50414) 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7165a23640 1 -- 172.21.15.50:0/169482184 learned_addr learned my addr 172.21.15.50:0/169482184 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7164a21640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71600c8370 0x7f716006e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.233+0000 7f7165222640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f716013eb90 0x7f71601f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7165a23640 1 -- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f716013eb90 msgr2=0x7f71601f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7165a23640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f716013eb90 0x7f71601f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7165a23640 1 -- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71600c8370 msgr2=0x7f716006e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7165a23640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71600c8370 0x7f716006e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7165a23640 1 -- 172.21.15.50:0/169482184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71601f3350 con 0x7f71600c68a0 2024-04-02T16:21:58.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7165222640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f716013eb90 0x7f71601f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7165a23640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71600c68a0 0x7f716006e7f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f715800e200 tx=0x7f7158009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7155ffb640 1 -- 172.21.15.50:0/169482184 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158039070 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7155ffb640 1 -- 172.21.15.50:0/169482184 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7158044070 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71601f3620 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7155ffb640 1 -- 172.21.15.50:0/169482184 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158009c10 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.234+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601f3be0 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.235+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71600756f0 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.236+0000 7f7155ffb640 1 -- 172.21.15.50:0/169482184 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7158005d10 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.236+0000 7f7155ffb640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f714406c920 0x7f714406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.236+0000 7f7155ffb640 1 -- 172.21.15.50:0/169482184 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f71580ad740 con 0x7f71600c68a0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.236+0000 7f7165222640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f714406c920 0x7f714406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.237+0000 7f7165222640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f714406c920 0x7f714406ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f715c001050 tx=0x7f715c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:58.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.240+0000 7f7155ffb640 1 -- 172.21.15.50:0/169482184 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f715804d020 con 0x7f71600c68a0 2024-04-02T16:21:58.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.424+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f716006c2d0 con 0x7f71600c68a0 2024-04-02T16:21:58.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.425+0000 7f7155ffb640 1 -- 172.21.15.50:0/169482184 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f715807c010 con 0x7f71600c68a0 2024-04-02T16:21:58.425 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:21:58.425 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-04-02T16:21:58.425 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:21:58.425 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-04-02T16:21:58.426 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-04-02T16:21:58.426 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:21:58.426 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-04-02T16:21:58.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f714406c920 msgr2=0x7f714406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f714406c920 0x7f714406ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f715c001050 tx=0x7f715c009040 comp rx=0 tx=0).stop 2024-04-02T16:21:58.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71600c68a0 msgr2=0x7f716006e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71600c68a0 0x7f716006e7f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f715800e200 tx=0x7f7158009750 comp rx=0 tx=0).stop 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 shutdown_connections 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f714406c920 0x7f714406ede0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71600c68a0 0x7f716006e7f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71600c8370 0x7f716006e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 --2- 172.21.15.50:0/169482184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f716013eb90 0x7f71601f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 >> 172.21.15.50:0/169482184 conn(0x7f71600b2d20 msgr2=0x7f71600cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 shutdown_connections 2024-04-02T16:21:58.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.427+0000 7f7167486640 1 -- 172.21.15.50:0/169482184 wait complete. 2024-04-02T16:21:58.437 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T16:21:58.438 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T16:21:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.542+0000 7f23893cd640 1 Processor -- start 2024-04-02T16:21:58.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23893cd640 1 -- start start 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23893cd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f23840d2890 0x7f23840cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23893cd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f238415e860 0x7f23840d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23893cd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23840d31b0 con 0x7f23840d4a50 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23893cd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f23840d01e0 con 0x7f238415e860 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23893cd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f23840cff00 con 0x7f23840d2890 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23827fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f238415e860 0x7f23840d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23827fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f238415e860 0x7f23840d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50418/0 (socket says 172.21.15.50:50418) 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.543+0000 7f23827fc640 1 -- 172.21.15.50:0/1636716428 learned_addr learned my addr 172.21.15.50:0/1636716428 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.544+0000 7f23827fc640 1 -- 172.21.15.50:0/1636716428 >> v2:172.21.15.188:3300/0 conn(0x7f23840d2890 msgr2=0x7f23840cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.544+0000 7f23827fc640 1 --2- 172.21.15.50:0/1636716428 >> v2:172.21.15.188:3300/0 conn(0x7f23840d2890 0x7f23840cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.544+0000 7f23827fc640 1 -- 172.21.15.50:0/1636716428 >> v1:172.21.15.188:6789/0 conn(0x7f23840d4a50 legacy=0x7f23840d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:58.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.544+0000 7f23827fc640 1 -- 172.21.15.50:0/1636716428 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23840b3d10 con 0x7f238415e860 2024-04-02T16:21:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.544+0000 7f23827fc640 1 --2- 172.21.15.50:0/1636716428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238415e860 0x7f23840d2c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f238409dcb0 tx=0x7f236c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa4fba4a07689ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f2381ffb640 1 -- 172.21.15.50:0/1636716428 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c02e070 con 0x7f238415e860 2024-04-02T16:21:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f2381ffb640 1 -- 172.21.15.50:0/1636716428 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c037070 con 0x7f238415e860 2024-04-02T16:21:58.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f2381ffb640 1 -- 172.21.15.50:0/1636716428 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c031a00 con 0x7f238415e860 2024-04-02T16:21:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 -- 172.21.15.50:0/1636716428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238415e860 msgr2=0x7f23840d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 --2- 172.21.15.50:0/1636716428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238415e860 0x7f23840d2c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f238409dcb0 tx=0x7f236c02b880 comp rx=0 tx=0).stop 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 -- 172.21.15.50:0/1636716428 shutdown_connections 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 --2- 172.21.15.50:0/1636716428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f238415e860 0x7f23840d2c70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 --2- 172.21.15.50:0/1636716428 >> v2:172.21.15.188:3300/0 conn(0x7f23840d2890 0x7f23840cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 -- 172.21.15.50:0/1636716428 >> 172.21.15.50:0/1636716428 conn(0x7f238409d580 msgr2=0x7f23840d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 -- 172.21.15.50:0/1636716428 shutdown_connections 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.545+0000 7f23893cd640 1 -- 172.21.15.50:0/1636716428 wait complete. 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.546+0000 7f23893cd640 1 Processor -- start 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.546+0000 7f23893cd640 1 -- start start 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23893cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23840d2890 0x7f23840b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23893cd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23840d4a50 0x7f23841524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23893cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f238415e860 0x7f2384152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23893cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f23840d4ec0 con 0x7f23840d2890 2024-04-02T16:21:58.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23893cd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f23840b42d0 con 0x7f238415e860 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23893cd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f23840d5040 con 0x7f23840d4a50 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23840d2890 0x7f23840b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23840d2890 0x7f23840b5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50424/0 (socket says 172.21.15.50:50424) 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f2382ffd640 1 -- 172.21.15.50:0/735232672 learned_addr learned my addr 172.21.15.50:0/735232672 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23827fc640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23840d4a50 0x7f23841524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.547+0000 7f23837fe640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f238415e860 0x7f2384152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f2382ffd640 1 -- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23840d4a50 msgr2=0x7f23841524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f2382ffd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23840d4a50 0x7f23841524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f2382ffd640 1 -- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f238415e860 msgr2=0x7f2384152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f2382ffd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f238415e860 0x7f2384152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f2382ffd640 1 -- 172.21.15.50:0/735232672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238415fa20 con 0x7f23840d2890 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f23827fc640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23840d4a50 0x7f23841524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f23837fe640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f238415e860 0x7f2384152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:21:58.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f2382ffd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23840d2890 0x7f23840b5d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f237c00b410 tx=0x7f237c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:58.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f237b7fe640 1 -- 172.21.15.50:0/735232672 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c015020 con 0x7f23840d2890 2024-04-02T16:21:58.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f237b7fe640 1 -- 172.21.15.50:0/735232672 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c011850 con 0x7f23840d2890 2024-04-02T16:21:58.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238415fd00 con 0x7f23840d2890 2024-04-02T16:21:58.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f237b7fe640 1 -- 172.21.15.50:0/735232672 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c011c40 con 0x7f23840d2890 2024-04-02T16:21:58.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.548+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23841605a0 con 0x7f23840d2890 2024-04-02T16:21:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.549+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23840757d0 con 0x7f23840d2890 2024-04-02T16:21:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.549+0000 7f237b7fe640 1 -- 172.21.15.50:0/735232672 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f237c018020 con 0x7f23840d2890 2024-04-02T16:21:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.550+0000 7f237b7fe640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.550+0000 7f237b7fe640 1 -- 172.21.15.50:0/735232672 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f237c08b790 con 0x7f23840d2890 2024-04-02T16:21:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.550+0000 7f23827fc640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:58.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.551+0000 7f23827fc640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f236406c8f0 0x7f236406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f236c036000 tx=0x7f236c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:58.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.554+0000 7f237b7fe640 1 -- 172.21.15.50:0/735232672 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f237c01f020 con 0x7f23840d2890 2024-04-02T16:21:58.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:58.744+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f23840d25c0 con 0x7f23840d2890 2024-04-02T16:21:59.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.368+0000 7f237b7fe640 1 -- 172.21.15.50:0/735232672 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f237c0172c0 con 0x7f23840d2890 2024-04-02T16:21:59.368 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-04-02T16:21:59.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f236406c8f0 msgr2=0x7f236406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:59.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f236406c8f0 0x7f236406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f236c036000 tx=0x7f236c03d040 comp rx=0 tx=0).stop 2024-04-02T16:21:59.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23840d2890 msgr2=0x7f23840b5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:59.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23840d2890 0x7f23840b5d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f237c00b410 tx=0x7f237c00b8d0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 shutdown_connections 2024-04-02T16:21:59.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f238415e860 0x7f2384152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23840d4a50 0x7f23841524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 --2- 172.21.15.50:0/735232672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23840d2890 0x7f23840b5d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.371+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 >> 172.21.15.50:0/735232672 conn(0x7f238409d580 msgr2=0x7f23840b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.373+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 shutdown_connections 2024-04-02T16:21:59.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.373+0000 7f23893cd640 1 -- 172.21.15.50:0/735232672 wait complete. 2024-04-02T16:21:59.389 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-02T16:21:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.503+0000 7fecd4338640 1 Processor -- start 2024-04-02T16:21:59.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.503+0000 7fecd4338640 1 -- start start 2024-04-02T16:21:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd4338640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd4338640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feccc0d13d0 0x7feccc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd4338640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc0b10e0 con 0x7feccc0d2e50 2024-04-02T16:21:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd4338640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7feccc0b1380 con 0x7feccc0d13d0 2024-04-02T16:21:59.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd4338640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feccc0b1660 con 0x7feccc161c80 2024-04-02T16:21:59.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd20d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:59.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd28d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feccc0d13d0 0x7feccc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:59.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd20d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:44718/0 (socket says 172.21.15.50:44718) 2024-04-02T16:21:59.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd20d4640 1 -- 172.21.15.50:0/1766893997 learned_addr learned my addr 172.21.15.50:0/1766893997 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd28d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feccc0d13d0 0x7feccc0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:57642/0 (socket says 172.21.15.50:57642) 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.504+0000 7fecd18d3640 1 --1- 172.21.15.50:0/1766893997 >> v1:172.21.15.188:6789/0 conn(0x7feccc0d2e50 0x7feccc0d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:42458/0 (socket says 172.21.15.50:42458) 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.505+0000 7fecd20d4640 1 -- 172.21.15.50:0/1766893997 >> v2:172.21.15.188:3300/0 conn(0x7feccc0d13d0 msgr2=0x7feccc0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.505+0000 7fecd20d4640 1 --2- 172.21.15.50:0/1766893997 >> v2:172.21.15.188:3300/0 conn(0x7feccc0d13d0 0x7feccc0b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.505+0000 7fecd20d4640 1 -- 172.21.15.50:0/1766893997 >> v1:172.21.15.188:6789/0 conn(0x7feccc0d2e50 legacy=0x7feccc0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.505+0000 7fecd20d4640 1 -- 172.21.15.50:0/1766893997 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc0d3340 con 0x7feccc161c80 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.505+0000 7fecd28d5640 1 --2- 172.21.15.50:0/1766893997 >> v2:172.21.15.188:3300/0 conn(0x7feccc0d13d0 0x7feccc0b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:21:59.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd20d4640 1 --2- 172.21.15.50:0/1766893997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0d2910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5047c0476c23841d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:59.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd10d2640 1 -- 172.21.15.50:0/1766893997 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4035020 con 0x7feccc161c80 2024-04-02T16:21:59.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd10d2640 1 -- 172.21.15.50:0/1766893997 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc4031640 con 0x7feccc161c80 2024-04-02T16:21:59.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd10d2640 1 -- 172.21.15.50:0/1766893997 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc402d040 con 0x7feccc161c80 2024-04-02T16:21:59.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 -- 172.21.15.50:0/1766893997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 msgr2=0x7feccc0d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:59.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 --2- 172.21.15.50:0/1766893997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0d2910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc402b8f0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 -- 172.21.15.50:0/1766893997 shutdown_connections 2024-04-02T16:21:59.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 --2- 172.21.15.50:0/1766893997 >> v2:172.21.15.188:3300/0 conn(0x7feccc0d13d0 0x7feccc0b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 --2- 172.21.15.50:0/1766893997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0d2910 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 -- 172.21.15.50:0/1766893997 >> 172.21.15.50:0/1766893997 conn(0x7feccc09d5e0 msgr2=0x7feccc0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:21:59.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 -- 172.21.15.50:0/1766893997 shutdown_connections 2024-04-02T16:21:59.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.506+0000 7fecd4338640 1 -- 172.21.15.50:0/1766893997 wait complete. 2024-04-02T16:21:59.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 Processor -- start 2024-04-02T16:21:59.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 -- start start 2024-04-02T16:21:59.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc0d13d0 0x7feccc0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feccc0d2e50 0x7feccc0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0b2520 con 0x7feccc0d2e50 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0b23a0 con 0x7feccc0d13d0 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd4338640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feccc0b26a0 con 0x7feccc161c80 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd18d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feccc0d2e50 0x7feccc0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd18d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feccc0d2e50 0x7feccc0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50426/0 (socket says 172.21.15.50:50426) 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.508+0000 7fecd18d3640 1 -- 172.21.15.50:0/3384426783 learned_addr learned my addr 172.21.15.50:0/3384426783 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd28d5640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd20d4640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc0d13d0 0x7feccc0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd18d3640 1 -- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 msgr2=0x7feccc0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd18d3640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd18d3640 1 -- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc0d13d0 msgr2=0x7feccc0b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:21:59.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd18d3640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc0d13d0 0x7feccc0b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd18d3640 1 -- 172.21.15.50:0/3384426783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc0b9060 con 0x7feccc0d2e50 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd18d3640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feccc0d2e50 0x7feccc0caa00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c440 tx=0x7fecbc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecc2ffd640 1 -- 172.21.15.50:0/3384426783 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0074e0 con 0x7feccc0d2e50 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecc2ffd640 1 -- 172.21.15.50:0/3384426783 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc007680 con 0x7feccc0d2e50 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc0b9340 con 0x7feccc0d2e50 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecc2ffd640 1 -- 172.21.15.50:0/3384426783 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc004a20 con 0x7feccc0d2e50 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.509+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc0caf40 con 0x7feccc0d2e50 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.510+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc075760 con 0x7feccc0d2e50 2024-04-02T16:21:59.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.511+0000 7fecc2ffd640 1 -- 172.21.15.50:0/3384426783 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecbc01a020 con 0x7feccc0d2e50 2024-04-02T16:21:59.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.511+0000 7fecc2ffd640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fecb006c8f0 0x7fecb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:21:59.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.511+0000 7fecc2ffd640 1 -- 172.21.15.50:0/3384426783 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fecbc08b030 con 0x7feccc0d2e50 2024-04-02T16:21:59.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.511+0000 7fecd20d4640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fecb006c8f0 0x7fecb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:21:59.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.512+0000 7fecd20d4640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fecb006c8f0 0x7fecb006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fecc4036000 tx=0x7fecc403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:21:59.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.516+0000 7fecc2ffd640 1 -- 172.21.15.50:0/3384426783 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fecbc059650 con 0x7feccc0d2e50 2024-04-02T16:21:59.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:21:59.699+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7feccc0d1c70 con 0x7feccc0d2e50 2024-04-02T16:22:00.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.370+0000 7fecc2ffd640 1 -- 172.21.15.50:0/3384426783 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7fecbc058f50 con 0x7feccc0d2e50 2024-04-02T16:22:00.371 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-04-02T16:22:00.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fecb006c8f0 msgr2=0x7fecb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:00.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fecb006c8f0 0x7fecb006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fecc4036000 tx=0x7fecc403c040 comp rx=0 tx=0).stop 2024-04-02T16:22:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feccc0d2e50 msgr2=0x7feccc0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feccc0d2e50 0x7feccc0caa00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c440 tx=0x7fecbc001e70 comp rx=0 tx=0).stop 2024-04-02T16:22:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 shutdown_connections 2024-04-02T16:22:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc161c80 0x7feccc0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fecb006c8f0 0x7fecb006edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feccc0d2e50 0x7feccc0caa00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 --2- 172.21.15.50:0/3384426783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc0d13d0 0x7feccc0b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.374+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 >> 172.21.15.50:0/3384426783 conn(0x7feccc09d5e0 msgr2=0x7feccc0cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.375+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 shutdown_connections 2024-04-02T16:22:00.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.375+0000 7fecd4338640 1 -- 172.21.15.50:0/3384426783 wait complete. 2024-04-02T16:22:00.402 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T16:22:00.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.510+0000 7fc48cbba640 1 Processor -- start 2024-04-02T16:22:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.510+0000 7fc48cbba640 1 -- start start 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48cbba640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc488161c80 0x7fc4880d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48cbba640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4880b10e0 con 0x7fc4880d2e50 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48cbba640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4880b1380 con 0x7fc4880d13d0 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48cbba640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fc4880b1660 con 0x7fc488161c80 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc488161c80 0x7fc4880d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc488161c80 0x7fc4880d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50434/0 (socket says 172.21.15.50:50434) 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48659c640 1 -- 172.21.15.50:0/3899153693 learned_addr learned my addr 172.21.15.50:0/3899153693 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc485d9b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc4880d2e50 0x7fc4880d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42342/0 (socket says 172.21.15.50:42342) 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.511+0000 7fc48659c640 1 -- 172.21.15.50:0/3899153693 >> v1:172.21.15.188:6789/0 conn(0x7fc4880d13d0 legacy=0x7fc4880b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.512+0000 7fc48659c640 1 -- 172.21.15.50:0/3899153693 >> v1:172.21.15.50:6789/0 conn(0x7fc4880d2e50 legacy=0x7fc4880d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.512+0000 7fc48559a640 1 -- 172.21.15.50:0/3899153693 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694541802 0 0) 0x7fc4880b10e0 con 0x7fc4880d2e50 2024-04-02T16:22:00.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.512+0000 7fc48659c640 1 -- 172.21.15.50:0/3899153693 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880d3340 con 0x7fc488161c80 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.512+0000 7fc48659c640 1 --2- 172.21.15.50:0/3899153693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc488161c80 0x7fc4880d2910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc478009e50 tx=0x7fc47802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beb879e987cf5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.512+0000 7fc48559a640 1 -- 172.21.15.50:0/3899153693 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47802ca80 con 0x7fc488161c80 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.512+0000 7fc48559a640 1 -- 172.21.15.50:0/3899153693 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47802cc20 con 0x7fc488161c80 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.512+0000 7fc48559a640 1 -- 172.21.15.50:0/3899153693 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4780343f0 con 0x7fc488161c80 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.513+0000 7fc48cbba640 1 -- 172.21.15.50:0/3899153693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc488161c80 msgr2=0x7fc4880d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.513+0000 7fc48cbba640 1 --2- 172.21.15.50:0/3899153693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc488161c80 0x7fc4880d2910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc478009e50 tx=0x7fc47802bee0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.513+0000 7fc48cbba640 1 -- 172.21.15.50:0/3899153693 shutdown_connections 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.513+0000 7fc48cbba640 1 --2- 172.21.15.50:0/3899153693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc488161c80 0x7fc4880d2910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.513+0000 7fc48cbba640 1 -- 172.21.15.50:0/3899153693 >> 172.21.15.50:0/3899153693 conn(0x7fc48809d5e0 msgr2=0x7fc4880d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.513+0000 7fc48cbba640 1 -- 172.21.15.50:0/3899153693 shutdown_connections 2024-04-02T16:22:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.513+0000 7fc48cbba640 1 -- 172.21.15.50:0/3899153693 wait complete. 2024-04-02T16:22:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.514+0000 7fc48cbba640 1 Processor -- start 2024-04-02T16:22:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.514+0000 7fc48cbba640 1 -- start start 2024-04-02T16:22:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48cbba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880d13d0 0x7fc4880ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48cbba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4880d2e50 0x7fc48814d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48cbba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc488161c80 0x7fc48814db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48cbba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880b26f0 con 0x7fc4880d13d0 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48cbba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880b2570 con 0x7fc4880d2e50 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48cbba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc4880b2870 con 0x7fc488161c80 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880d13d0 0x7fc4880ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880d13d0 0x7fc4880ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50436/0 (socket says 172.21.15.50:50436) 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 -- 172.21.15.50:0/2524496165 learned_addr learned my addr 172.21.15.50:0/2524496165 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc485d9b640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4880d2e50 0x7fc48814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc486d9d640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc488161c80 0x7fc48814db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 -- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc488161c80 msgr2=0x7fc48814db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc488161c80 0x7fc48814db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 -- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4880d2e50 msgr2=0x7fc48814d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:00.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4880d2e50 0x7fc48814d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:00.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.515+0000 7fc48659c640 1 -- 172.21.15.50:0/2524496165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880b81f0 con 0x7fc4880d13d0 2024-04-02T16:22:00.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.516+0000 7fc48659c640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880d13d0 0x7fc4880ca850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc478009e50 tx=0x7fc478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:00.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.516+0000 7fc47effd640 1 -- 172.21.15.50:0/2524496165 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478043020 con 0x7fc4880d13d0 2024-04-02T16:22:00.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.516+0000 7fc47effd640 1 -- 172.21.15.50:0/2524496165 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47803e450 con 0x7fc4880d13d0 2024-04-02T16:22:00.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.516+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4880b84d0 con 0x7fc4880d13d0 2024-04-02T16:22:00.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.516+0000 7fc47effd640 1 -- 172.21.15.50:0/2524496165 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47803e840 con 0x7fc4880d13d0 2024-04-02T16:22:00.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.516+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4880b8a90 con 0x7fc4880d13d0 2024-04-02T16:22:00.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.517+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc488075760 con 0x7fc4880d13d0 2024-04-02T16:22:00.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.517+0000 7fc47effd640 1 -- 172.21.15.50:0/2524496165 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc478046020 con 0x7fc4880d13d0 2024-04-02T16:22:00.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.518+0000 7fc47effd640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:00.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.518+0000 7fc47effd640 1 -- 172.21.15.50:0/2524496165 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fc4780ae4a0 con 0x7fc4880d13d0 2024-04-02T16:22:00.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.518+0000 7fc485d9b640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:00.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.518+0000 7fc485d9b640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc470000f60 tx=0x7fc470007bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:00.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.522+0000 7fc47effd640 1 -- 172.21.15.50:0/2524496165 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc47804d030 con 0x7fc4880d13d0 2024-04-02T16:22:00.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:00.746+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc4880cf8d0 con 0x7fc4880d13d0 2024-04-02T16:22:01.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.404+0000 7fc47effd640 1 -- 172.21.15.50:0/2524496165 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fc47807c920 con 0x7fc4880d13d0 2024-04-02T16:22:01.404 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 2 and data pool 3 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc46806c8f0 msgr2=0x7fc46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc46806c8f0 0x7fc46806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc470000f60 tx=0x7fc470007bd0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880d13d0 msgr2=0x7fc4880ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880d13d0 0x7fc4880ca850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc478009e50 tx=0x7fc478004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 shutdown_connections 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc488161c80 0x7fc48814db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4880d2e50 0x7fc48814d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc46806c8f0 0x7fc46806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 --2- 172.21.15.50:0/2524496165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4880d13d0 0x7fc4880ca850 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.409+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 >> 172.21.15.50:0/2524496165 conn(0x7fc48809d5e0 msgr2=0x7fc48806d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:01.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.410+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 shutdown_connections 2024-04-02T16:22:01.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.410+0000 7fc48cbba640 1 -- 172.21.15.50:0/2524496165 wait complete. 2024-04-02T16:22:01.428 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-02T16:22:01.428 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-02T16:22:01.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.540+0000 7f6607559640 1 Processor -- start 2024-04-02T16:22:01.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.540+0000 7f6607559640 1 -- start start 2024-04-02T16:22:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6607559640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000cd6a0 0x7f66000d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6607559640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66000d0e20 0x7f660006b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6607559640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f660015c840 con 0x7f660015c460 2024-04-02T16:22:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6607559640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f660006edc0 con 0x7f66000d0e20 2024-04-02T16:22:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6607559640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f66000cf5d0 con 0x7f66000cd6a0 2024-04-02T16:22:01.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6605af6640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f66000d0e20 0x7f660006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f66052f5640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f660015c460 0x7f66000ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42352/0 (socket says 172.21.15.50:42352) 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f66052f5640 1 -- 172.21.15.50:0/78179641 learned_addr learned my addr 172.21.15.50:0/78179641 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6604af4640 1 --2- 172.21.15.50:0/78179641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000cd6a0 0x7f66000d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6605af6640 1 -- 172.21.15.50:0/78179641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000cd6a0 msgr2=0x7f66000d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6605af6640 1 --2- 172.21.15.50:0/78179641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000cd6a0 0x7f66000d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6605af6640 1 -- 172.21.15.50:0/78179641 >> v1:172.21.15.50:6789/0 conn(0x7f660015c460 legacy=0x7f66000ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6605af6640 1 -- 172.21.15.50:0/78179641 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66000cf8b0 con 0x7f66000d0e20 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f65f7fff640 1 -- 172.21.15.50:0/78179641 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 240789716 0 0) 0x7f660015c840 con 0x7f660015c460 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.541+0000 7f6604af4640 1 --2- 172.21.15.50:0/78179641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000cd6a0 0x7f66000d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.542+0000 7f6605af6640 1 --2- 172.21.15.50:0/78179641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000d0e20 0x7f660006b520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009d90 tx=0x7f65f802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6098253647070eef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.542+0000 7f65f7fff640 1 -- 172.21.15.50:0/78179641 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8039070 con 0x7f66000d0e20 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.542+0000 7f65f7fff640 1 -- 172.21.15.50:0/78179641 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f802cc50 con 0x7f66000d0e20 2024-04-02T16:22:01.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.542+0000 7f65f7fff640 1 -- 172.21.15.50:0/78179641 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8034440 con 0x7f66000d0e20 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 -- 172.21.15.50:0/78179641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000d0e20 msgr2=0x7f660006b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 --2- 172.21.15.50:0/78179641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000d0e20 0x7f660006b520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009d90 tx=0x7f65f802bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 -- 172.21.15.50:0/78179641 shutdown_connections 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 --2- 172.21.15.50:0/78179641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000d0e20 0x7f660006b520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 --2- 172.21.15.50:0/78179641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66000cd6a0 0x7f66000d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 -- 172.21.15.50:0/78179641 >> 172.21.15.50:0/78179641 conn(0x7f66000b2b40 msgr2=0x7f66000d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 -- 172.21.15.50:0/78179641 shutdown_connections 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.543+0000 7f6607559640 1 -- 172.21.15.50:0/78179641 wait complete. 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 Processor -- start 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 -- start start 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000cd6a0 0x7f660015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:01.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000d0e20 0x7f6600151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f660015c460 0x7f66001519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f66000afa80 con 0x7f66000cd6a0 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f660006ea40 con 0x7f66000d0e20 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.544+0000 7f6607559640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f66000afc00 con 0x7f660015c460 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000cd6a0 0x7f660015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000cd6a0 0x7f660015ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50462/0 (socket says 172.21.15.50:50462) 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 -- 172.21.15.50:0/677931363 learned_addr learned my addr 172.21.15.50:0/677931363 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f6604af4640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000d0e20 0x7f6600151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f6605af6640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f660015c460 0x7f66001519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 -- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f660015c460 msgr2=0x7f66001519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f660015c460 0x7f66001519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 -- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000d0e20 msgr2=0x7f6600151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000d0e20 0x7f6600151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:01.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 -- 172.21.15.50:0/677931363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66001539b0 con 0x7f66000cd6a0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f6604af4640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000d0e20 0x7f6600151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f6605af6640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f660015c460 0x7f66001519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f66052f5640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000cd6a0 0x7f660015ee70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c4d0 tx=0x7f65fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f65f5ffb640 1 -- 172.21.15.50:0/677931363 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc018070 con 0x7f66000cd6a0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f65f5ffb640 1 -- 172.21.15.50:0/677931363 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc007500 con 0x7f66000cd6a0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f65f5ffb640 1 -- 172.21.15.50:0/677931363 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc004a20 con 0x7f66000cd6a0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.545+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6600153c90 con 0x7f66000cd6a0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.546+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6600154250 con 0x7f66000cd6a0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.546+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66000757d0 con 0x7f66000cd6a0 2024-04-02T16:22:01.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.547+0000 7f65f5ffb640 1 -- 172.21.15.50:0/677931363 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65fc01b070 con 0x7f66000cd6a0 2024-04-02T16:22:01.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.547+0000 7f65f5ffb640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f65e406c8f0 0x7f65e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:01.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.547+0000 7f65f5ffb640 1 -- 172.21.15.50:0/677931363 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f65fc08ae00 con 0x7f66000cd6a0 2024-04-02T16:22:01.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.547+0000 7f6604af4640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f65e406c8f0 0x7f65e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:01.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.548+0000 7f6604af4640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f65e406c8f0 0x7f65e406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006fd0 tx=0x7f65f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:01.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.552+0000 7f65f5ffb640 1 -- 172.21.15.50:0/677931363 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f65fc0592c0 con 0x7f66000cd6a0 2024-04-02T16:22:01.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:01.748+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f660015b830 con 0x7f66000cd6a0 2024-04-02T16:22:02.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:22:02.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:22:02.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.388+0000 7f65f5ffb640 1 -- 172.21.15.50:0/677931363 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f65fc058bc0 con 0x7f66000cd6a0 2024-04-02T16:22:02.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f65e406c8f0 msgr2=0x7f65e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:02.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f65e406c8f0 0x7f65e406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006fd0 tx=0x7f65f0008040 comp rx=0 tx=0).stop 2024-04-02T16:22:02.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000cd6a0 msgr2=0x7f660015ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:02.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000cd6a0 0x7f660015ee70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c4d0 tx=0x7f65fc00c990 comp rx=0 tx=0).stop 2024-04-02T16:22:02.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 shutdown_connections 2024-04-02T16:22:02.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f660015c460 0x7f66001519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66000d0e20 0x7f6600151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f65e406c8f0 0x7f65e406edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 --2- 172.21.15.50:0/677931363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66000cd6a0 0x7f660015ee70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 >> 172.21.15.50:0/677931363 conn(0x7f66000b2b40 msgr2=0x7f660015c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:02.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 shutdown_connections 2024-04-02T16:22:02.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.391+0000 7f6607559640 1 -- 172.21.15.50:0/677931363 wait complete. 2024-04-02T16:22:02.404 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T16:22:02.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.516+0000 7f7866752640 1 Processor -- start 2024-04-02T16:22:02.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.516+0000 7f7866752640 1 -- start start 2024-04-02T16:22:02.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f7866752640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78600b46e0 0x7f78600cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f7866752640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78600ccc00 con 0x7f786009aa40 2024-04-02T16:22:02.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f7866752640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78600ccea0 con 0x7f786015c300 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f7866752640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f78600cd180 con 0x7f78600b46e0 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f7864cef640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f786009aa40 0x7f786006f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42356/0 (socket says 172.21.15.50:42356) 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f7864cef640 1 -- 172.21.15.50:0/1995390204 learned_addr learned my addr 172.21.15.50:0/1995390204 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f785f7fe640 1 --2- 172.21.15.50:0/1995390204 >> v2:172.21.15.188:3300/0 conn(0x7f78600b46e0 0x7f78600cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.517+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721151693 0 0) 0x7f78600ccc00 con 0x7f786009aa40 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7848003520 con 0x7f786009aa40 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2583791413 0 0) 0x7f7848003520 con 0x7f786009aa40 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 >> v2:172.21.15.188:3300/0 conn(0x7f78600b46e0 msgr2=0x7f78600cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 --2- 172.21.15.50:0/1995390204 >> v2:172.21.15.188:3300/0 conn(0x7f78600b46e0 0x7f78600cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 >> v1:172.21.15.188:6789/0 conn(0x7f786015c300 legacy=0x7f78600cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78600cd460 con 0x7f786009aa40 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f78580019f0 con 0x7f786009aa40 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785f7fe640 1 --2- 172.21.15.50:0/1995390204 >> v2:172.21.15.188:3300/0 conn(0x7f78600b46e0 0x7f78600cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 >> v1:172.21.15.50:6789/0 conn(0x7f786009aa40 legacy=0x7f786006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78480039b0 0x7f7848003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7848004800 0x7f7848004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78480051c0 0x7f784800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78600cd180 con 0x7f78480051c0 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f784800c060 con 0x7f7848004800 2024-04-02T16:22:02.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f784800c260 con 0x7f78480039b0 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f7864cef640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78480051c0 0x7f784800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785ffff640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78480039b0 0x7f7848003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f785f7fe640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7848004800 0x7f7848004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f7864cef640 1 -- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78480039b0 msgr2=0x7f7848003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f7864cef640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78480039b0 0x7f7848003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f7864cef640 1 -- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7848004800 msgr2=0x7f7848004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f7864cef640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7848004800 0x7f7848004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.518+0000 7f7864cef640 1 -- 172.21.15.50:0/1995390204 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7848003520 con 0x7f78480051c0 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.519+0000 7f7864cef640 1 -- 172.21.15.50:0/1995390204 reap_dead start 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.519+0000 7f7864cef640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78480051c0 0x7f784800b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f785800a850 tx=0x7f7848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70b7a53a8edd5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:02.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.519+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785800a4a0 con 0x7f78480051c0 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.519+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7858034650 con 0x7f78480051c0 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.519+0000 7f785effd640 1 -- 172.21.15.50:0/1995390204 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858034a40 con 0x7f78480051c0 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.520+0000 7f7866752640 1 -- 172.21.15.50:0/1995390204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78480051c0 msgr2=0x7f784800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.520+0000 7f7866752640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78480051c0 0x7f784800b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f785800a850 tx=0x7f7848005830 comp rx=0 tx=0).stop 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.520+0000 7f7866752640 1 -- 172.21.15.50:0/1995390204 shutdown_connections 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.520+0000 7f7866752640 1 --2- 172.21.15.50:0/1995390204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78480051c0 0x7f784800b9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.520+0000 7f7866752640 1 -- 172.21.15.50:0/1995390204 >> 172.21.15.50:0/1995390204 conn(0x7f78600b3b50 msgr2=0x7f78600b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.520+0000 7f7866752640 1 -- 172.21.15.50:0/1995390204 shutdown_connections 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.520+0000 7f7866752640 1 -- 172.21.15.50:0/1995390204 wait complete. 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.521+0000 7f7866752640 1 Processor -- start 2024-04-02T16:22:02.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.521+0000 7f7866752640 1 -- start start 2024-04-02T16:22:02.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78480051c0 0x7f78601530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78600cb290 0x7f786015d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78600af990 0x7f78600afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78600cdc60 con 0x7f78600af990 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f786015c6e0 con 0x7f78480051c0 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f78600cdde0 con 0x7f78600cb290 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78600af990 0x7f78600afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78600af990 0x7f78600afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50468/0 (socket says 172.21.15.50:50468) 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 -- 172.21.15.50:0/1468761057 learned_addr learned my addr 172.21.15.50:0/1468761057 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:02.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f785f7fe640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78600cb290 0x7f786015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f785ffff640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78480051c0 0x7f78601530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 -- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78600cb290 msgr2=0x7f786015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78600cb290 0x7f786015d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 -- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78480051c0 msgr2=0x7f78601530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78480051c0 0x7f78601530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 -- 172.21.15.50:0/1468761057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786015da60 con 0x7f78600af990 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f785ffff640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78480051c0 0x7f78601530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7864cef640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78600af990 0x7f78600afe40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f785800a850 tx=0x7f7858001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f785cff9640 1 -- 172.21.15.50:0/1468761057 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858039040 con 0x7f78600af990 2024-04-02T16:22:02.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f785cff9640 1 -- 172.21.15.50:0/1468761057 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7858043070 con 0x7f78600af990 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786015dd40 con 0x7f78600af990 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f785cff9640 1 -- 172.21.15.50:0/1468761057 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858004d60 con 0x7f78600af990 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.522+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786015e300 con 0x7f78600af990 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.524+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78600756f0 con 0x7f78600af990 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.524+0000 7f785cff9640 1 -- 172.21.15.50:0/1468761057 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7858034650 con 0x7f78600af990 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.524+0000 7f785cff9640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f784c06c920 0x7f784c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.524+0000 7f785cff9640 1 -- 172.21.15.50:0/1468761057 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f78580ad8f0 con 0x7f78600af990 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.524+0000 7f785ffff640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f784c06c920 0x7f784c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.525+0000 7f785ffff640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f784c06c920 0x7f784c06ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7850005580 tx=0x7f7850009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.528+0000 7f785cff9640 1 -- 172.21.15.50:0/1468761057 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f78580019b0 con 0x7f78600af990 2024-04-02T16:22:02.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:02.714+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f78600c55f0 con 0x7f78600af990 2024-04-02T16:22:04.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.417+0000 7f785cff9640 1 -- 172.21.15.50:0/1468761057 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f785807bc80 con 0x7f78600af990 2024-04-02T16:22:04.417 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' created 2024-04-02T16:22:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f784c06c920 msgr2=0x7f784c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f784c06c920 0x7f784c06ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7850005580 tx=0x7f7850009040 comp rx=0 tx=0).stop 2024-04-02T16:22:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78600af990 msgr2=0x7f78600afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78600af990 0x7f78600afe40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f785800a850 tx=0x7f7858001dc0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 shutdown_connections 2024-04-02T16:22:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f784c06c920 0x7f784c06ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78600af990 0x7f78600afe40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78600cb290 0x7f786015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 --2- 172.21.15.50:0/1468761057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78480051c0 0x7f78601530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.420+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 >> 172.21.15.50:0/1468761057 conn(0x7f78600b3b50 msgr2=0x7f786006f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.421+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 shutdown_connections 2024-04-02T16:22:04.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.421+0000 7f7866752640 1 -- 172.21.15.50:0/1468761057 wait complete. 2024-04-02T16:22:04.442 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-02T16:22:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.552+0000 7f24408a5640 1 Processor -- start 2024-04-02T16:22:04.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f24408a5640 1 -- start start 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f24408a5640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f243c162570 0x7f243c0cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f24408a5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f243c0d0110 0x7f243c15e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f24408a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c162950 0x7f243c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f24408a5640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f243c15ebd0 con 0x7f243c162570 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f24408a5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f243c0d0670 con 0x7f243c0d0110 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f24408a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f243c0d0950 con 0x7f243c162950 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f243a59c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f243c162570 0x7f243c0cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f243a59c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f243c162570 0x7f243c0cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50470/0 (socket says 172.21.15.50:50470) 2024-04-02T16:22:04.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.553+0000 7f243a59c640 1 -- 172.21.15.50:0/38393876 learned_addr learned my addr 172.21.15.50:0/38393876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f2433fff640 1 --2- 172.21.15.50:0/38393876 >> v2:172.21.15.188:3300/0 conn(0x7f243c0d0110 0x7f243c15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f243ad9d640 1 --2- 172.21.15.50:0/38393876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c162950 0x7f243c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f243a59c640 1 -- 172.21.15.50:0/38393876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c162950 msgr2=0x7f243c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f243a59c640 1 --2- 172.21.15.50:0/38393876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c162950 0x7f243c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f243a59c640 1 -- 172.21.15.50:0/38393876 >> v2:172.21.15.188:3300/0 conn(0x7f243c0d0110 msgr2=0x7f243c15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f243a59c640 1 --2- 172.21.15.50:0/38393876 >> v2:172.21.15.188:3300/0 conn(0x7f243c0d0110 0x7f243c15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f243a59c640 1 -- 172.21.15.50:0/38393876 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0d0c30 con 0x7f243c162570 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.554+0000 7f243a59c640 1 --2- 172.21.15.50:0/38393876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c0cfbd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f243c08cb50 tx=0x7f242c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a34b0ca69ed3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f2439d9b640 1 -- 172.21.15.50:0/38393876 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c035020 con 0x7f243c162570 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f2439d9b640 1 -- 172.21.15.50:0/38393876 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242c031870 con 0x7f243c162570 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f2439d9b640 1 -- 172.21.15.50:0/38393876 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c031c40 con 0x7f243c162570 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 -- 172.21.15.50:0/38393876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 msgr2=0x7f243c0cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:04.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 --2- 172.21.15.50:0/38393876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c0cfbd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f243c08cb50 tx=0x7f242c02b910 comp rx=0 tx=0).stop 2024-04-02T16:22:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 -- 172.21.15.50:0/38393876 shutdown_connections 2024-04-02T16:22:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 --2- 172.21.15.50:0/38393876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c162950 0x7f243c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 --2- 172.21.15.50:0/38393876 >> v2:172.21.15.188:3300/0 conn(0x7f243c0d0110 0x7f243c15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 --2- 172.21.15.50:0/38393876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c0cfbd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 -- 172.21.15.50:0/38393876 >> 172.21.15.50:0/38393876 conn(0x7f243c09d5a0 msgr2=0x7f243c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 -- 172.21.15.50:0/38393876 shutdown_connections 2024-04-02T16:22:04.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.555+0000 7f24408a5640 1 -- 172.21.15.50:0/38393876 wait complete. 2024-04-02T16:22:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.556+0000 7f24408a5640 1 Processor -- start 2024-04-02T16:22:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f24408a5640 1 -- start start 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f24408a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0d0110 0x7f243c0b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f24408a5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f24408a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c162950 0x7f243c0a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f24408a5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f243c162fe0 con 0x7f243c162570 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f24408a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f243c0d11f0 con 0x7f243c162950 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f24408a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f243c163160 con 0x7f243c0d0110 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f2433fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f2433fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c1f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50476/0 (socket says 172.21.15.50:50476) 2024-04-02T16:22:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f2433fff640 1 -- 172.21.15.50:0/4128307572 learned_addr learned my addr 172.21.15.50:0/4128307572 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f243a59c640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0d0110 0x7f243c0b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.557+0000 7f243ad9d640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c162950 0x7f243c0a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2433fff640 1 -- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0d0110 msgr2=0x7f243c0b94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2433fff640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0d0110 0x7f243c0b94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2433fff640 1 -- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c162950 msgr2=0x7f243c0a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2433fff640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c162950 0x7f243c0a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2433fff640 1 -- 172.21.15.50:0/4128307572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0b8400 con 0x7f243c162570 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2433fff640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c1f38e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f242400e840 tx=0x7f242400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2432ffd640 1 -- 172.21.15.50:0/4128307572 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424017070 con 0x7f243c162570 2024-04-02T16:22:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2432ffd640 1 -- 172.21.15.50:0/4128307572 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242400c940 con 0x7f243c162570 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c0b86e0 con 0x7f243c162570 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f2432ffd640 1 -- 172.21.15.50:0/4128307572 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242400cd10 con 0x7f243c162570 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.558+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c0b8f80 con 0x7f243c162570 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.559+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c0b88b0 con 0x7f243c162570 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.560+0000 7f2432ffd640 1 -- 172.21.15.50:0/4128307572 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f242401a070 con 0x7f243c162570 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.560+0000 7f2432ffd640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f241c06c8f0 0x7f241c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.560+0000 7f2432ffd640 1 -- 172.21.15.50:0/4128307572 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f242408bfe0 con 0x7f243c162570 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.560+0000 7f243a59c640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f241c06c8f0 0x7f241c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.561+0000 7f243a59c640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f241c06c8f0 0x7f241c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f242c036000 tx=0x7f242c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:04.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.565+0000 7f2432ffd640 1 -- 172.21.15.50:0/4128307572 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2424059aa0 con 0x7f243c162570 2024-04-02T16:22:04.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:04.749+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f243c0757d0 con 0x7f243c162570 2024-04-02T16:22:05.237 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-04-02T16:22:05.236+0000 7fb450c82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T16:22:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.422+0000 7f2432ffd640 1 -- 172.21.15.50:0/4128307572 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f242405dda0 con 0x7f243c162570 2024-04-02T16:22:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:set pool 4 allow_ec_overwrites to true 2024-04-02T16:22:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.427+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f241c06c8f0 msgr2=0x7f241c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.427+0000 7f24408a5640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f241c06c8f0 0x7f241c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f242c036000 tx=0x7f242c03c040 comp rx=0 tx=0).stop 2024-04-02T16:22:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.427+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 msgr2=0x7f243c1f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.427+0000 7f24408a5640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c1f38e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f242400e840 tx=0x7f242400ed00 comp rx=0 tx=0).stop 2024-04-02T16:22:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.428+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 shutdown_connections 2024-04-02T16:22:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.428+0000 7f24408a5640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c162950 0x7f243c0a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.428+0000 7f24408a5640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f241c06c8f0 0x7f241c06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.428+0000 7f24408a5640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f243c162570 0x7f243c1f38e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.428+0000 7f24408a5640 1 --2- 172.21.15.50:0/4128307572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0d0110 0x7f243c0b94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.428+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 >> 172.21.15.50:0/4128307572 conn(0x7f243c09d5a0 msgr2=0x7f243c15fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:05.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.432+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 shutdown_connections 2024-04-02T16:22:05.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.433+0000 7f24408a5640 1 -- 172.21.15.50:0/4128307572 wait complete. 2024-04-02T16:22:05.459 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-02T16:22:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.566+0000 7fb354eb7640 1 Processor -- start 2024-04-02T16:22:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.566+0000 7fb354eb7640 1 -- start start 2024-04-02T16:22:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb354eb7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb35015c720 0x7fb3500b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb354eb7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35006f630 0x7fb3500cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb354eb7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3500ce110 con 0x7fb35015c340 2024-04-02T16:22:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb354eb7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb3500d0710 con 0x7fb35015c720 2024-04-02T16:22:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb354eb7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3500cd030 con 0x7fb35006f630 2024-04-02T16:22:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb345d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb35015c720 0x7fb3500b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb34e59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb35015c340 0x7fb3500cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42368/0 (socket says 172.21.15.50:42368) 2024-04-02T16:22:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.567+0000 7fb34e59c640 1 -- 172.21.15.50:0/583882414 learned_addr learned my addr 172.21.15.50:0/583882414 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.568+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/583882414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35006f630 0x7fb3500cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.568+0000 7fb345d9b640 1 -- 172.21.15.50:0/583882414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35006f630 msgr2=0x7fb3500cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.568+0000 7fb345d9b640 1 --2- 172.21.15.50:0/583882414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35006f630 0x7fb3500cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.568+0000 7fb345d9b640 1 -- 172.21.15.50:0/583882414 >> v1:172.21.15.50:6789/0 conn(0x7fb35015c340 legacy=0x7fb3500cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.568+0000 7fb34dd9b640 1 -- 172.21.15.50:0/583882414 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683997181 0 0) 0x7fb340001110 con 0x7fb35015c340 2024-04-02T16:22:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.568+0000 7fb345d9b640 1 -- 172.21.15.50:0/583882414 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500d0960 con 0x7fb35015c720 2024-04-02T16:22:05.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.568+0000 7fb345d9b640 1 --2- 172.21.15.50:0/583882414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500b4740 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3380097a0 tx=0x7fb33802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bd1da41cd5d03a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb34dd9b640 1 -- 172.21.15.50:0/583882414 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338035020 con 0x7fb35015c720 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb34dd9b640 1 -- 172.21.15.50:0/583882414 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb338031830 con 0x7fb35015c720 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb34dd9b640 1 -- 172.21.15.50:0/583882414 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338031c00 con 0x7fb35015c720 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 -- 172.21.15.50:0/583882414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 msgr2=0x7fb3500b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 --2- 172.21.15.50:0/583882414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500b4740 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3380097a0 tx=0x7fb33802b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 -- 172.21.15.50:0/583882414 shutdown_connections 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 --2- 172.21.15.50:0/583882414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35006f630 0x7fb3500cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 --2- 172.21.15.50:0/583882414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500b4740 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 -- 172.21.15.50:0/583882414 >> 172.21.15.50:0/583882414 conn(0x7fb3500b3c20 msgr2=0x7fb3500b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 -- 172.21.15.50:0/583882414 shutdown_connections 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.569+0000 7fb354eb7640 1 -- 172.21.15.50:0/583882414 wait complete. 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 Processor -- start 2024-04-02T16:22:05.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 -- start start 2024-04-02T16:22:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb35006f630 0x7fb3500c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35015c340 0x7fb3500c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb3500d0f20 con 0x7fb35015c720 2024-04-02T16:22:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb35006fae0 con 0x7fb35006f630 2024-04-02T16:22:05.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb354eb7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3500ccc20 con 0x7fb35015c340 2024-04-02T16:22:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb34ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb34ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50502/0 (socket says 172.21.15.50:50502) 2024-04-02T16:22:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.571+0000 7fb34ed9d640 1 -- 172.21.15.50:0/3186275304 learned_addr learned my addr 172.21.15.50:0/3186275304 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb345d9b640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35015c340 0x7fb3500c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:05.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb34e59c640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb35006f630 0x7fb3500c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:05.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb34ed9d640 1 -- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35015c340 msgr2=0x7fb3500c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:05.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35015c340 0x7fb3500c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb34ed9d640 1 -- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb35006f630 msgr2=0x7fb3500c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:05.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb35006f630 0x7fb3500c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:05.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb34ed9d640 1 -- 172.21.15.50:0/3186275304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500c8920 con 0x7fb35015c720 2024-04-02T16:22:05.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb345d9b640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35015c340 0x7fb3500c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb34ed9d640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500cb770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb34800cc30 tx=0x7fb34800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb3477fe640 1 -- 172.21.15.50:0/3186275304 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34800ce40 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb3477fe640 1 -- 172.21.15.50:0/3186275304 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb348003990 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb3477fe640 1 -- 172.21.15.50:0/3186275304 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348003d40 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35006dcb0 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.572+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35006e550 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.573+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35006de80 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.574+0000 7fb3477fe640 1 -- 172.21.15.50:0/3186275304 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb348003ee0 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.574+0000 7fb3477fe640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb33006c8f0 0x7fb33006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.574+0000 7fb3477fe640 1 -- 172.21.15.50:0/3186275304 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fb34808c2a0 con 0x7fb35015c720 2024-04-02T16:22:05.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.574+0000 7fb34e59c640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb33006c8f0 0x7fb33006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:05.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.575+0000 7fb34e59c640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb33006c8f0 0x7fb33006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb340000f60 tx=0x7fb340007970 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:05.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.578+0000 7fb3477fe640 1 -- 172.21.15.50:0/3186275304 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb348014030 con 0x7fb35015c720 2024-04-02T16:22:05.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:05.802+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fb350075760 con 0x7fb35015c720 2024-04-02T16:22:06.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.424+0000 7fb3477fe640 1 -- 172.21.15.50:0/3186275304 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fb34805a440 con 0x7fb35015c720 2024-04-02T16:22:06.425 INFO:teuthology.orchestra.run.smithi050.stderr:added data pool 4 to fsmap 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb33006c8f0 msgr2=0x7fb33006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb33006c8f0 0x7fb33006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb340000f60 tx=0x7fb340007970 comp rx=0 tx=0).stop 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 msgr2=0x7fb3500cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500cb770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb34800cc30 tx=0x7fb34800a5c0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 shutdown_connections 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb33006c8f0 0x7fb33006edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35015c720 0x7fb3500cb770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb35015c340 0x7fb3500c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 --2- 172.21.15.50:0/3186275304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb35006f630 0x7fb3500c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 >> 172.21.15.50:0/3186275304 conn(0x7fb3500b3c20 msgr2=0x7fb3500b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 shutdown_connections 2024-04-02T16:22:06.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.427+0000 7fb354eb7640 1 -- 172.21.15.50:0/3186275304 wait complete. 2024-04-02T16:22:06.441 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:22:06.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.548+0000 7f43df9e2640 1 Processor -- start 2024-04-02T16:22:06.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.548+0000 7f43df9e2640 1 -- start start 2024-04-02T16:22:06.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43df9e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43df9e2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43d80d15a0 0x7f43d80d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43df9e2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d80ceda0 con 0x7f43d80d2800 2024-04-02T16:22:06.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43df9e2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f43d80d31c0 con 0x7f43d80d15a0 2024-04-02T16:22:06.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43df9e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f43d80cf040 con 0x7f43d815e510 2024-04-02T16:22:06.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43dd77e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43dd77e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57548/0 (socket says 172.21.15.50:57548) 2024-04-02T16:22:06.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43dd77e640 1 -- 172.21.15.50:0/1788869675 learned_addr learned my addr 172.21.15.50:0/1788869675 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:06.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.549+0000 7f43ddf7f640 1 --2- 172.21.15.50:0/1788869675 >> v2:172.21.15.188:3300/0 conn(0x7f43d80d15a0 0x7f43d80d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.550+0000 7f43dd77e640 1 -- 172.21.15.50:0/1788869675 >> v2:172.21.15.188:3300/0 conn(0x7f43d80d15a0 msgr2=0x7f43d80d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:06.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.550+0000 7f43dd77e640 1 --2- 172.21.15.50:0/1788869675 >> v2:172.21.15.188:3300/0 conn(0x7f43d80d15a0 0x7f43d80d2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.550+0000 7f43dd77e640 1 -- 172.21.15.50:0/1788869675 >> v1:172.21.15.188:6789/0 conn(0x7f43d80d2800 legacy=0x7f43d80d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.550+0000 7f43dd77e640 1 -- 172.21.15.50:0/1788869675 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d80cf320 con 0x7f43d815e510 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.550+0000 7f43ddf7f640 1 --2- 172.21.15.50:0/1788869675 >> v2:172.21.15.188:3300/0 conn(0x7f43d80d15a0 0x7f43d80d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.550+0000 7f43dd77e640 1 --2- 172.21.15.50:0/1788869675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80cfe00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009870 tx=0x7f43d002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79df945f238f875d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43cffff640 1 -- 172.21.15.50:0/1788869675 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0035020 con 0x7f43d815e510 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43cffff640 1 -- 172.21.15.50:0/1788869675 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d0031640 con 0x7f43d815e510 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43cffff640 1 -- 172.21.15.50:0/1788869675 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d002d040 con 0x7f43d815e510 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 -- 172.21.15.50:0/1788869675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 msgr2=0x7f43d80cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1788869675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80cfe00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009870 tx=0x7f43d002b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 -- 172.21.15.50:0/1788869675 shutdown_connections 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1788869675 >> v2:172.21.15.188:3300/0 conn(0x7f43d80d15a0 0x7f43d80d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1788869675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80cfe00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 -- 172.21.15.50:0/1788869675 >> 172.21.15.50:0/1788869675 conn(0x7f43d809d570 msgr2=0x7f43d80d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:06.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 -- 172.21.15.50:0/1788869675 shutdown_connections 2024-04-02T16:22:06.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.551+0000 7f43df9e2640 1 -- 172.21.15.50:0/1788869675 wait complete. 2024-04-02T16:22:06.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.552+0000 7f43df9e2640 1 Processor -- start 2024-04-02T16:22:06.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.552+0000 7f43df9e2640 1 -- start start 2024-04-02T16:22:06.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43df9e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d80d15a0 0x7f43d8152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43df9e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43d80d2800 0x7f43d80cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43df9e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43df9e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80b3ad0 con 0x7f43d80d2800 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43df9e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43d815e8f0 con 0x7f43d80d15a0 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43df9e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f43d80b3c50 con 0x7f43d815e510 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43d80d2800 0x7f43d80cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43d80d2800 0x7f43d80cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60128/0 (socket says 172.21.15.50:60128) 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 -- 172.21.15.50:0/1212952171 learned_addr learned my addr 172.21.15.50:0/1212952171 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43ddf7f640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dd77e640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d80d15a0 0x7f43d8152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 -- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 msgr2=0x7f43d80ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 -- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d80d15a0 msgr2=0x7f43d8152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d80d15a0 0x7f43d8152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.553+0000 7f43dcf7d640 1 -- 172.21.15.50:0/1212952171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d8160c10 con 0x7f43d80d2800 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.554+0000 7f43dcf7d640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43d80d2800 0x7f43d80cc7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c450 tx=0x7f43c800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.554+0000 7f43cdffb640 1 -- 172.21.15.50:0/1212952171 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8017070 con 0x7f43d80d2800 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.554+0000 7f43cdffb640 1 -- 172.21.15.50:0/1212952171 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c80052d0 con 0x7f43d80d2800 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.554+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d8161960 con 0x7f43d80d2800 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.554+0000 7f43cdffb640 1 -- 172.21.15.50:0/1212952171 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8003830 con 0x7f43d80d2800 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.554+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d8160e80 con 0x7f43d80d2800 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.555+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d80756f0 con 0x7f43d80d2800 2024-04-02T16:22:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.555+0000 7f43cdffb640 1 -- 172.21.15.50:0/1212952171 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43c801a070 con 0x7f43d80d2800 2024-04-02T16:22:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.556+0000 7f43cdffb640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f43bc06c8f0 0x7f43bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.556+0000 7f43cdffb640 1 -- 172.21.15.50:0/1212952171 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f43c808b110 con 0x7f43d80d2800 2024-04-02T16:22:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.556+0000 7f43dd77e640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f43bc06c8f0 0x7f43bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.556+0000 7f43dd77e640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f43bc06c8f0 0x7f43bc06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43d0036000 tx=0x7f43d003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.560+0000 7f43cdffb640 1 -- 172.21.15.50:0/1212952171 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f43c8058bb0 con 0x7f43d80d2800 2024-04-02T16:22:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.786+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43d80ce6c0 con 0x7f43d80d2800 2024-04-02T16:22:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.787+0000 7f43cdffb640 1 -- 172.21.15.50:0/1212952171 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5177 (secure 0 0 0) 0x7f43c805ceb0 con 0x7f43d80d2800 2024-04-02T16:22:06.787 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:06.787 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:05.802975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:22:06.788 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 7 2024-04-02T16:22:06.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f43bc06c8f0 msgr2=0x7f43bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f43bc06c8f0 0x7f43bc06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43d0036000 tx=0x7f43d003c040 comp rx=0 tx=0).stop 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43d80d2800 msgr2=0x7f43d80cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43d80d2800 0x7f43d80cc7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c450 tx=0x7f43c800c850 comp rx=0 tx=0).stop 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 shutdown_connections 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43d815e510 0x7f43d80ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f43bc06c8f0 0x7f43bc06edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43d80d2800 0x7f43d80cc7c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 --2- 172.21.15.50:0/1212952171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43d80d15a0 0x7f43d8152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 >> 172.21.15.50:0/1212952171 conn(0x7f43d809d570 msgr2=0x7f43d806d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 shutdown_connections 2024-04-02T16:22:06.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.789+0000 7f43df9e2640 1 -- 172.21.15.50:0/1212952171 wait complete. 2024-04-02T16:22:06.800 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:06.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.901+0000 7f621d0eb640 1 Processor -- start 2024-04-02T16:22:06.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.901+0000 7f621d0eb640 1 -- start start 2024-04-02T16:22:06.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f621d0eb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6218145680 0x7f62180cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f621d0eb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621806f900 con 0x7f621806b520 2024-04-02T16:22:06.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f621d0eb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62180d0d80 con 0x7f6218072790 2024-04-02T16:22:06.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f621d0eb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f62180cffb0 con 0x7f6218145680 2024-04-02T16:22:06.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f6216d9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f621806b520 0x7f621806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49456/0 (socket says 172.21.15.50:49456) 2024-04-02T16:22:06.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f6216d9d640 1 -- 172.21.15.50:0/1861801578 learned_addr learned my addr 172.21.15.50:0/1861801578 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:06.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f620e59c640 1 --2- 172.21.15.50:0/1861801578 >> v2:172.21.15.188:3300/0 conn(0x7f6218145680 0x7f62180cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4106483279 0 0) 0x7f6208001110 con 0x7f621806b520 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6204003520 con 0x7f621806b520 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.902+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666473475 0 0) 0x7f62180d0d80 con 0x7f6218072790 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.903+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6208001110 con 0x7f6218072790 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.903+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4229065270 0 0) 0x7f6204003520 con 0x7f621806b520 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.903+0000 7f620e59c640 1 -- 172.21.15.50:0/1861801578 >> v1:172.21.15.188:6789/0 conn(0x7f6218072790 legacy=0x7f6218072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.903+0000 7f620e59c640 1 -- 172.21.15.50:0/1861801578 >> v1:172.21.15.50:6789/0 conn(0x7f621806b520 legacy=0x7f621806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.903+0000 7f620e59c640 1 -- 172.21.15.50:0/1861801578 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6218145e70 con 0x7f6218145680 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.903+0000 7f620e59c640 1 --2- 172.21.15.50:0/1861801578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6218145680 0x7f62180cfa70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62000097d0 tx=0x7f620002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f61273de92ac5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200037070 con 0x7f6218145680 2024-04-02T16:22:06.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6200030510 con 0x7f6218145680 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621659c640 1 -- 172.21.15.50:0/1861801578 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208001840 con 0x7f6218145680 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621d0eb640 1 -- 172.21.15.50:0/1861801578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6218145680 msgr2=0x7f62180cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621d0eb640 1 --2- 172.21.15.50:0/1861801578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6218145680 0x7f62180cfa70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62000097d0 tx=0x7f620002b790 comp rx=0 tx=0).stop 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621d0eb640 1 -- 172.21.15.50:0/1861801578 shutdown_connections 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621d0eb640 1 --2- 172.21.15.50:0/1861801578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6218145680 0x7f62180cfa70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621d0eb640 1 -- 172.21.15.50:0/1861801578 >> 172.21.15.50:0/1861801578 conn(0x7f62180b2d20 msgr2=0x7f62180b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621d0eb640 1 -- 172.21.15.50:0/1861801578 shutdown_connections 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.904+0000 7f621d0eb640 1 -- 172.21.15.50:0/1861801578 wait complete. 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.905+0000 7f621d0eb640 1 Processor -- start 2024-04-02T16:22:06.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621d0eb640 1 -- start start 2024-04-02T16:22:06.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621d0eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621806b520 0x7f62181f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621d0eb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6218072790 0x7f62181f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621d0eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6218145680 0x7f62180b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621d0eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f62180d07c0 con 0x7f6218145680 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621d0eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62180d04e0 con 0x7f621806b520 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621d0eb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f62180d0aa0 con 0x7f6218072790 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6218145680 0x7f62180b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6218145680 0x7f62180b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60138/0 (socket says 172.21.15.50:60138) 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f621759e640 1 -- 172.21.15.50:0/4292530802 learned_addr learned my addr 172.21.15.50:0/4292530802 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.906+0000 7f6216d9d640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621806b520 0x7f62181f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f620e59c640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6218072790 0x7f62181f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621759e640 1 -- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6218072790 msgr2=0x7f62181f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621759e640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6218072790 0x7f62181f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621759e640 1 -- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621806b520 msgr2=0x7f62181f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621759e640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621806b520 0x7f62181f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621759e640 1 -- 172.21.15.50:0/4292530802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62180b5910 con 0x7f6218145680 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621759e640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6218145680 0x7f62180b4ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f621000c790 tx=0x7f621000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:06.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f620ffff640 1 -- 172.21.15.50:0/4292530802 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210003b70 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f620ffff640 1 -- 172.21.15.50:0/4292530802 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6210003d10 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62181f50b0 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f620ffff640 1 -- 172.21.15.50:0/4292530802 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621000aa50 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.907+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181f5670 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.908+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62180756f0 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.909+0000 7f620ffff640 1 -- 172.21.15.50:0/4292530802 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f621000abf0 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.909+0000 7f620ffff640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f61f806c8f0 0x7f61f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.909+0000 7f620ffff640 1 -- 172.21.15.50:0/4292530802 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f621008bc70 con 0x7f6218145680 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.909+0000 7f6216d9d640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f61f806c8f0 0x7f61f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:06.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.910+0000 7f6216d9d640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f61f806c8f0 0x7f61f806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62080013b0 tx=0x7f6208001dd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:06.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:06.914+0000 7f620ffff640 1 -- 172.21.15.50:0/4292530802 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f62100597a0 con 0x7f6218145680 2024-04-02T16:22:07.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.100+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f621806c2d0 con 0x7f6218145680 2024-04-02T16:22:07.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.101+0000 7f620ffff640 1 -- 172.21.15.50:0/4292530802 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18515 (secure 0 0 0) 0x7f621005daa0 con 0x7f6218145680 2024-04-02T16:22:07.102 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:07.102 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:05.802776+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f61f806c8f0 msgr2=0x7f61f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f61f806c8f0 0x7f61f806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62080013b0 tx=0x7f6208001dd0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6218145680 msgr2=0x7f62180b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6218145680 0x7f62180b4ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f621000c790 tx=0x7f621000cc50 comp rx=0 tx=0).stop 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 shutdown_connections 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f61f806c8f0 0x7f61f806edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6218145680 0x7f62180b4ba0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6218072790 0x7f62181f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 --2- 172.21.15.50:0/4292530802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621806b520 0x7f62181f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 >> 172.21.15.50:0/4292530802 conn(0x7f62180b2d20 msgr2=0x7f6218072c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 shutdown_connections 2024-04-02T16:22:07.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.103+0000 7f621d0eb640 1 -- 172.21.15.50:0/4292530802 wait complete. 2024-04-02T16:22:07.114 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:07.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.215+0000 7f931f56d640 1 Processor -- start 2024-04-02T16:22:07.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.216+0000 7f931f56d640 1 -- start start 2024-04-02T16:22:07.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.216+0000 7f931f56d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93180d2f10 0x7f93180cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.216+0000 7f931f56d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f93180d0e90 0x7f93180d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.216+0000 7f931f56d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93180ceda0 con 0x7f931815e510 2024-04-02T16:22:07.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.216+0000 7f931f56d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f93180cf040 con 0x7f93180d0e90 2024-04-02T16:22:07.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.216+0000 7f931f56d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f93180cf320 con 0x7f93180d2f10 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931cb08640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93180d2f10 0x7f93180cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931db0a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f93180d0e90 0x7f93180d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931d309640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f931815e510 0x7f93180d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:40256/0 (socket says 172.21.15.50:40256) 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931d309640 1 -- 172.21.15.50:0/1976418416 learned_addr learned my addr 172.21.15.50:0/1976418416 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931cb08640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93180d2f10 0x7f93180cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57580/0 (socket says 172.21.15.50:57580) 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931db0a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f93180d0e90 0x7f93180d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:39276/0 (socket says 172.21.15.50:39276) 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931cb08640 1 -- 172.21.15.50:0/1976418416 >> v2:172.21.15.188:3300/0 conn(0x7f93180d0e90 msgr2=0x7f93180d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931cb08640 1 --2- 172.21.15.50:0/1976418416 >> v2:172.21.15.188:3300/0 conn(0x7f93180d0e90 0x7f93180d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931cb08640 1 -- 172.21.15.50:0/1976418416 >> v1:172.21.15.188:6789/0 conn(0x7f931815e510 legacy=0x7f93180d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931cb08640 1 -- 172.21.15.50:0/1976418416 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93180cf600 con 0x7f93180d2f10 2024-04-02T16:22:07.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.217+0000 7f931db0a640 1 --2- 172.21.15.50:0/1976418416 >> v2:172.21.15.188:3300/0 conn(0x7f93180d0e90 0x7f93180d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f931cb08640 1 --2- 172.21.15.50:0/1976418416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93180d2f10 0x7f93180cfe00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097a0 tx=0x7f930c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb4e7723cbdf48e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f9317fff640 1 -- 172.21.15.50:0/1976418416 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c0315a0 con 0x7f93180d2f10 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f9317fff640 1 -- 172.21.15.50:0/1976418416 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930c037070 con 0x7f93180d2f10 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f9317fff640 1 -- 172.21.15.50:0/1976418416 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c031960 con 0x7f93180d2f10 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f931f56d640 1 -- 172.21.15.50:0/1976418416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93180d2f10 msgr2=0x7f93180cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f931f56d640 1 --2- 172.21.15.50:0/1976418416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93180d2f10 0x7f93180cfe00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097a0 tx=0x7f930c02b840 comp rx=0 tx=0).stop 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f931f56d640 1 -- 172.21.15.50:0/1976418416 shutdown_connections 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f931f56d640 1 --2- 172.21.15.50:0/1976418416 >> v2:172.21.15.188:3300/0 conn(0x7f93180d0e90 0x7f93180d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f931f56d640 1 --2- 172.21.15.50:0/1976418416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93180d2f10 0x7f93180cfe00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.218+0000 7f931f56d640 1 -- 172.21.15.50:0/1976418416 >> 172.21.15.50:0/1976418416 conn(0x7f931809d570 msgr2=0x7f93180d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.219+0000 7f931f56d640 1 -- 172.21.15.50:0/1976418416 shutdown_connections 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.219+0000 7f931f56d640 1 -- 172.21.15.50:0/1976418416 wait complete. 2024-04-02T16:22:07.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 Processor -- start 2024-04-02T16:22:07.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 -- start start 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93180d0e90 0x7f93180cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93180d2f10 0x7f93180c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f931815e510 0x7f93180c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f93180d19e0 con 0x7f93180d0e90 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93180d1860 con 0x7f93180d2f10 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.220+0000 7f931f56d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f93180b3ad0 con 0x7f931815e510 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93180d0e90 0x7f93180cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93180d0e90 0x7f93180cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60150/0 (socket says 172.21.15.50:60150) 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 -- 172.21.15.50:0/1459853765 learned_addr learned my addr 172.21.15.50:0/1459853765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:07.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931cb08640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93180d2f10 0x7f93180c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931db0a640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f931815e510 0x7f93180c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 -- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f931815e510 msgr2=0x7f93180c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f931815e510 0x7f93180c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 -- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93180d2f10 msgr2=0x7f93180c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93180d2f10 0x7f93180c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 -- 172.21.15.50:0/1459853765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931814aad0 con 0x7f93180d0e90 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931d309640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93180d0e90 0x7f93180cd180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f931000c3c0 tx=0x7f931000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f9315ffb640 1 -- 172.21.15.50:0/1459853765 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310017070 con 0x7f93180d0e90 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f9315ffb640 1 -- 172.21.15.50:0/1459853765 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9310005ce0 con 0x7f93180d0e90 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93180cd6c0 con 0x7f93180d0e90 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f9315ffb640 1 -- 172.21.15.50:0/1459853765 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93100072f0 con 0x7f93180d0e90 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.221+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93180c9e70 con 0x7f93180d0e90 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.223+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93180756f0 con 0x7f93180d0e90 2024-04-02T16:22:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.223+0000 7f9315ffb640 1 -- 172.21.15.50:0/1459853765 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f931001a070 con 0x7f93180d0e90 2024-04-02T16:22:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.223+0000 7f9315ffb640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f930406c8f0 0x7f930406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.223+0000 7f9315ffb640 1 -- 172.21.15.50:0/1459853765 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f931008b090 con 0x7f93180d0e90 2024-04-02T16:22:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.223+0000 7f931cb08640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f930406c8f0 0x7f930406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.224+0000 7f931cb08640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f930406c8f0 0x7f930406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f930c036000 tx=0x7f930c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.228+0000 7f9315ffb640 1 -- 172.21.15.50:0/1459853765 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f931814aad0 con 0x7f93180d0e90 2024-04-02T16:22:07.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.412+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93180c9b20 con 0x7f93180d0e90 2024-04-02T16:22:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.413+0000 7f9315ffb640 1 -- 172.21.15.50:0/1459853765 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18515 (secure 0 0 0) 0x7f9310058b30 con 0x7f93180d0e90 2024-04-02T16:22:07.414 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:07.414 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":24,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:05.802776+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f930406c8f0 msgr2=0x7f930406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f930406c8f0 0x7f930406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f930c036000 tx=0x7f930c03d040 comp rx=0 tx=0).stop 2024-04-02T16:22:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93180d0e90 msgr2=0x7f93180cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93180d0e90 0x7f93180cd180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f931000c3c0 tx=0x7f931000c7c0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 shutdown_connections 2024-04-02T16:22:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f931815e510 0x7f93180c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93180d2f10 0x7f93180c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f930406c8f0 0x7f930406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 --2- 172.21.15.50:0/1459853765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93180d0e90 0x7f93180cd180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 >> 172.21.15.50:0/1459853765 conn(0x7f931809d570 msgr2=0x7f93180b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 shutdown_connections 2024-04-02T16:22:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.416+0000 7f931f56d640 1 -- 172.21.15.50:0/1459853765 wait complete. 2024-04-02T16:22:07.442 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:22:07.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.544+0000 7fb60220d640 1 Processor -- start 2024-04-02T16:22:07.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.544+0000 7fb60220d640 1 -- start start 2024-04-02T16:22:07.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb60220d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb5fc0d2f10 0x7fb5fc0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb60220d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5fc0d0e90 0x7fb5fc0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb60220d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5fc0ceda0 con 0x7fb5fc15e510 2024-04-02T16:22:07.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb60220d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb5fc0cf040 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb60220d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb5fc0cf320 con 0x7fb5fc0d0e90 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5faffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb5fc0d2f10 0x7fb5fc0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5faffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb5fc0d2f10 0x7fb5fc0cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60158/0 (socket says 172.21.15.50:60158) 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5faffd640 1 -- 172.21.15.50:0/3289611483 learned_addr learned my addr 172.21.15.50:0/3289611483 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5fbfff640 1 --2- 172.21.15.50:0/3289611483 >> v2:172.21.15.188:3300/0 conn(0x7fb5fc0d0e90 0x7fb5fc0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5faffd640 1 -- 172.21.15.50:0/3289611483 >> v2:172.21.15.188:3300/0 conn(0x7fb5fc0d0e90 msgr2=0x7fb5fc0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5faffd640 1 --2- 172.21.15.50:0/3289611483 >> v2:172.21.15.188:3300/0 conn(0x7fb5fc0d0e90 0x7fb5fc0d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5faffd640 1 -- 172.21.15.50:0/3289611483 >> v1:172.21.15.188:6789/0 conn(0x7fb5fc15e510 legacy=0x7fb5fc0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5faffd640 1 -- 172.21.15.50:0/3289611483 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc0cf600 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.545+0000 7fb5fbfff640 1 --2- 172.21.15.50:0/3289611483 >> v2:172.21.15.188:3300/0 conn(0x7fb5fc0d0e90 0x7fb5fc0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.546+0000 7fb5faffd640 1 --2- 172.21.15.50:0/3289611483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cfe00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0a21b0 tx=0x7fb5f002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af08509259a50294 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.546+0000 7fb5fa7fc640 1 -- 172.21.15.50:0/3289611483 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00317c0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.546+0000 7fb5fa7fc640 1 -- 172.21.15.50:0/3289611483 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f0037070 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.546+0000 7fb5fa7fc640 1 -- 172.21.15.50:0/3289611483 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0031b80 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 -- 172.21.15.50:0/3289611483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 msgr2=0x7fb5fc0cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 --2- 172.21.15.50:0/3289611483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cfe00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0a21b0 tx=0x7fb5f002b860 comp rx=0 tx=0).stop 2024-04-02T16:22:07.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 -- 172.21.15.50:0/3289611483 shutdown_connections 2024-04-02T16:22:07.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 --2- 172.21.15.50:0/3289611483 >> v2:172.21.15.188:3300/0 conn(0x7fb5fc0d0e90 0x7fb5fc0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 --2- 172.21.15.50:0/3289611483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cfe00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 -- 172.21.15.50:0/3289611483 >> 172.21.15.50:0/3289611483 conn(0x7fb5fc09d570 msgr2=0x7fb5fc0d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:07.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 -- 172.21.15.50:0/3289611483 shutdown_connections 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.547+0000 7fb60220d640 1 -- 172.21.15.50:0/3289611483 wait complete. 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.548+0000 7fb60220d640 1 Processor -- start 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.548+0000 7fb60220d640 1 -- start start 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5fc0d0e90 0x7fb5fc15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5fc15e510 0x7fb5fc0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc0d1970 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc0d17f0 con 0x7fb5fc0d0e90 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb5fc0b3ad0 con 0x7fb5fc15e510 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60174/0 (socket says 172.21.15.50:60174) 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 -- 172.21.15.50:0/374985519 learned_addr learned my addr 172.21.15.50:0/374985519 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5fb7fe640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5fc0d0e90 0x7fb5fc15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5fbfff640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5fc15e510 0x7fb5fc0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 -- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5fc15e510 msgr2=0x7fb5fc0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5fc15e510 0x7fb5fc0cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 -- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5fc0d0e90 msgr2=0x7fb5fc15ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5fc0d0e90 0x7fb5fc15ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 -- 172.21.15.50:0/374985519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc15ef10 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5fb7fe640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5fc0d0e90 0x7fb5fc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5faffd640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cd580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5f002b8a0 tx=0x7fb5f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5e3fff640 1 -- 172.21.15.50:0/374985519 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0032860 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5e3fff640 1 -- 172.21.15.50:0/374985519 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f0032a00 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5fc15f1f0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb5e3fff640 1 -- 172.21.15.50:0/374985519 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0032dd0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.549+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5fc15f7b0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.551+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5fc0756f0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.551+0000 7fb5e3fff640 1 -- 172.21.15.50:0/374985519 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5f00453f0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.551+0000 7fb5e3fff640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb5e806c8f0 0x7fb5e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.551+0000 7fb5e3fff640 1 -- 172.21.15.50:0/374985519 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb5f00ae030 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.551+0000 7fb5fb7fe640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb5e806c8f0 0x7fb5e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.552+0000 7fb5fb7fe640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb5e806c8f0 0x7fb5e806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0b4f00 tx=0x7fb5ec008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.555+0000 7fb5e3fff640 1 -- 172.21.15.50:0/374985519 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb5f0040070 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.778+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5fc1604f0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.779+0000 7fb5e3fff640 1 -- 172.21.15.50:0/374985519 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5177 (secure 0 0 0) 0x7fb5f007c1b0 con 0x7fb5fc0d2f10 2024-04-02T16:22:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:07.779 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:05.802975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:22:07.780 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 7 2024-04-02T16:22:07.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb5e806c8f0 msgr2=0x7fb5e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb5e806c8f0 0x7fb5e806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0b4f00 tx=0x7fb5ec008040 comp rx=0 tx=0).stop 2024-04-02T16:22:07.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 msgr2=0x7fb5fc0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cd580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5f002b8a0 tx=0x7fb5f00046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 shutdown_connections 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5fc15e510 0x7fb5fc0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb5e806c8f0 0x7fb5e806edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5fc0d2f10 0x7fb5fc0cd580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 --2- 172.21.15.50:0/374985519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5fc0d0e90 0x7fb5fc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 >> 172.21.15.50:0/374985519 conn(0x7fb5fc09d570 msgr2=0x7fb5fc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 shutdown_connections 2024-04-02T16:22:07.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.781+0000 7fb60220d640 1 -- 172.21.15.50:0/374985519 wait complete. 2024-04-02T16:22:07.792 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.893+0000 7f6ca92c7640 1 Processor -- start 2024-04-02T16:22:07.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.893+0000 7f6ca92c7640 1 -- start start 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6ca92c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b3b40 0x7f6ca40b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6ca92c7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ca40b4a50 0x7f6ca4142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6ca92c7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca40b5400 con 0x7f6ca40b1bc0 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6ca92c7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca40b4ee0 con 0x7f6ca40b4a50 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6ca92c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca40b5120 con 0x7f6ca40b3b40 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b3b40 0x7f6ca40b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6ca37fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ca40b4a50 0x7f6ca4142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b3b40 0x7f6ca40b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57602/0 (socket says 172.21.15.50:57602) 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6c9bfff640 1 -- 172.21.15.50:0/882004565 learned_addr learned my addr 172.21.15.50:0/882004565 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6c9bfff640 1 -- 172.21.15.50:0/882004565 >> v2:172.21.15.188:3300/0 conn(0x7f6ca40b4a50 msgr2=0x7f6ca4142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6c9bfff640 1 --2- 172.21.15.50:0/882004565 >> v2:172.21.15.188:3300/0 conn(0x7f6ca40b4a50 0x7f6ca4142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6c9bfff640 1 -- 172.21.15.50:0/882004565 >> v1:172.21.15.188:6789/0 conn(0x7f6ca40b1bc0 legacy=0x7f6ca40b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6c9bfff640 1 -- 172.21.15.50:0/882004565 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca40b3060 con 0x7f6ca40b3b40 2024-04-02T16:22:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.894+0000 7f6ca37fe640 1 --2- 172.21.15.50:0/882004565 >> v2:172.21.15.188:3300/0 conn(0x7f6ca40b4a50 0x7f6ca4142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.895+0000 7f6c9bfff640 1 --2- 172.21.15.50:0/882004565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b3b40 0x7f6ca40b4510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009870 tx=0x7f6c8c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72d8abd612c2e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.895+0000 7f6ca27fc640 1 -- 172.21.15.50:0/882004565 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c035020 con 0x7f6ca40b3b40 2024-04-02T16:22:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.895+0000 7f6ca27fc640 1 -- 172.21.15.50:0/882004565 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c031640 con 0x7f6ca40b3b40 2024-04-02T16:22:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.895+0000 7f6ca27fc640 1 -- 172.21.15.50:0/882004565 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c02d040 con 0x7f6ca40b3b40 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 -- 172.21.15.50:0/882004565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b3b40 msgr2=0x7f6ca40b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/882004565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b3b40 0x7f6ca40b4510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009870 tx=0x7f6c8c02b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 -- 172.21.15.50:0/882004565 shutdown_connections 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/882004565 >> v2:172.21.15.188:3300/0 conn(0x7f6ca40b4a50 0x7f6ca4142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/882004565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b3b40 0x7f6ca40b4510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 -- 172.21.15.50:0/882004565 >> 172.21.15.50:0/882004565 conn(0x7f6ca406ebf0 msgr2=0x7f6ca406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 -- 172.21.15.50:0/882004565 shutdown_connections 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.896+0000 7f6ca92c7640 1 -- 172.21.15.50:0/882004565 wait complete. 2024-04-02T16:22:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.897+0000 7f6ca92c7640 1 Processor -- start 2024-04-02T16:22:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.897+0000 7f6ca92c7640 1 -- start start 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca92c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b1bc0 0x7f6ca414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca92c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca40b3b40 0x7f6ca41382a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca92c7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca40b4a50 0x7f6ca415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca92c7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4142b00 con 0x7f6ca40b4a50 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca92c7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca40b3620 con 0x7f6ca40b3b40 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca92c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca4142c80 con 0x7f6ca40b1bc0 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca40b4a50 0x7f6ca415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca40b4a50 0x7f6ca415d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60176/0 (socket says 172.21.15.50:60176) 2024-04-02T16:22:07.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 -- 172.21.15.50:0/2106001127 learned_addr learned my addr 172.21.15.50:0/2106001127 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6c9bfff640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca40b3b40 0x7f6ca41382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca2ffd640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b1bc0 0x7f6ca414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 -- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b1bc0 msgr2=0x7f6ca414d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b1bc0 0x7f6ca414d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 -- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca40b3b40 msgr2=0x7f6ca41382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca40b3b40 0x7f6ca41382a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 -- 172.21.15.50:0/2106001127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca41387e0 con 0x7f6ca40b4a50 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6c9bfff640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca40b3b40 0x7f6ca41382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca2ffd640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b1bc0 0x7f6ca414d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.898+0000 7f6ca37fe640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca40b4a50 0x7f6ca415d8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009f00 tx=0x7f6c9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.899+0000 7f6c9b7fe640 1 -- 172.21.15.50:0/2106001127 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c018030 con 0x7f6ca40b4a50 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.899+0000 7f6c9b7fe640 1 -- 172.21.15.50:0/2106001127 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9c00ce50 con 0x7f6ca40b4a50 2024-04-02T16:22:07.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.899+0000 7f6c9b7fe640 1 -- 172.21.15.50:0/2106001127 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003bb0 con 0x7f6ca40b4a50 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.899+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca4071330 con 0x7f6ca40b4a50 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.899+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca40718f0 con 0x7f6ca40b4a50 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.900+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca40756f0 con 0x7f6ca40b4a50 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.900+0000 7f6c9b7fe640 1 -- 172.21.15.50:0/2106001127 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c9c005d10 con 0x7f6ca40b4a50 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.900+0000 7f6c9b7fe640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6c8406c8f0 0x7f6c8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.900+0000 7f6c9b7fe640 1 -- 172.21.15.50:0/2106001127 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6c9c08c8e0 con 0x7f6ca40b4a50 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.901+0000 7f6ca2ffd640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6c8406c8f0 0x7f6c8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:07.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.901+0000 7f6ca2ffd640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6c8406c8f0 0x7f6c8406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c94001660 tx=0x7f6c94009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:07.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:07.905+0000 7f6c9b7fe640 1 -- 172.21.15.50:0/2106001127 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6c9c022070 con 0x7f6ca40b4a50 2024-04-02T16:22:08.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.090+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ca406c280 con 0x7f6ca40b4a50 2024-04-02T16:22:08.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.091+0000 7f6c9b7fe640 1 -- 172.21.15.50:0/2106001127 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f6c9c010040 con 0x7f6ca40b4a50 2024-04-02T16:22:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:08.093 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:08.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6c8406c8f0 msgr2=0x7f6c8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6c8406c8f0 0x7f6c8406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c94001660 tx=0x7f6c94009040 comp rx=0 tx=0).stop 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca40b4a50 msgr2=0x7f6ca415d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca40b4a50 0x7f6ca415d8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009f00 tx=0x7f6c9c00c490 comp rx=0 tx=0).stop 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 shutdown_connections 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6c8406c8f0 0x7f6c8406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca40b4a50 0x7f6ca415d8c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca40b3b40 0x7f6ca41382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 --2- 172.21.15.50:0/2106001127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca40b1bc0 0x7f6ca414d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 >> 172.21.15.50:0/2106001127 conn(0x7f6ca406ebf0 msgr2=0x7f6ca40b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 shutdown_connections 2024-04-02T16:22:08.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.093+0000 7f6ca92c7640 1 -- 172.21.15.50:0/2106001127 wait complete. 2024-04-02T16:22:08.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T16:22:00.747313+0000', 'modified': '2024-04-02T16:22:05.802975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6837/2647829095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 2647829095}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 2647829095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-04-02T16:22:08.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T16:22:08.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T16:22:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.612+0000 7f6b3b33d640 1 Processor -- start 2024-04-02T16:22:08.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.612+0000 7f6b3b33d640 1 -- start start 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b3b33d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6b340c7e30 0x7f6b340d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b3b33d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b340d2f30 con 0x7f6b3415e7b0 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b3b33d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b340cf000 con 0x7f6b3415e3d0 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b3b33d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6b340cf2e0 con 0x7f6b340c7e30 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b398da640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f6b3415e7b0 0x7f6b340d26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49470/0 (socket says 172.21.15.50:49470) 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b398da640 1 -- 172.21.15.50:0/3363917538 learned_addr learned my addr 172.21.15.50:0/3363917538 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b388d8640 1 --2- 172.21.15.50:0/3363917538 >> v2:172.21.15.188:3300/0 conn(0x7f6b340c7e30 0x7f6b340d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818233548 0 0) 0x7f6b340d2f30 con 0x7f6b3415e7b0 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b20003520 con 0x7f6b3415e7b0 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 52126203 0 0) 0x7f6b20003520 con 0x7f6b3415e7b0 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 >> v2:172.21.15.188:3300/0 conn(0x7f6b340c7e30 msgr2=0x7f6b340d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 --2- 172.21.15.50:0/3363917538 >> v2:172.21.15.188:3300/0 conn(0x7f6b340c7e30 0x7f6b340d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 >> v1:172.21.15.188:6789/0 conn(0x7f6b3415e3d0 legacy=0x7f6b340d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b340cf5c0 con 0x7f6b3415e7b0 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f6b2c0019f0 con 0x7f6b3415e7b0 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b388d8640 1 --2- 172.21.15.50:0/3363917538 >> v2:172.21.15.188:3300/0 conn(0x7f6b340c7e30 0x7f6b340d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 >> v1:172.21.15.50:6789/0 conn(0x7f6b3415e7b0 legacy=0x7f6b340d26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b20004800 0x7f6b20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b200051c0 0x7f6b2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340cf2e0 con 0x7f6b200051c0 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2000c060 con 0x7f6b20004800 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.613+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2000c260 con 0x7f6b200039b0 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b398da640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b200051c0 0x7f6b2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b390d9640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b388d8640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b20004800 0x7f6b20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b398da640 1 -- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200039b0 msgr2=0x7f6b20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b398da640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b398da640 1 -- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b20004800 msgr2=0x7f6b20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b398da640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b20004800 0x7f6b20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b398da640 1 -- 172.21.15.50:0/3363917538 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b20003520 con 0x7f6b200051c0 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.614+0000 7f6b398da640 1 -- 172.21.15.50:0/3363917538 reap_dead start 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b398da640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b200051c0 0x7f6b2000b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00a850 tx=0x7f6b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841399e7356e04cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00a4a0 con 0x7f6b200051c0 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c034650 con 0x7f6b200051c0 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b2bfff640 1 -- 172.21.15.50:0/3363917538 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c034a40 con 0x7f6b200051c0 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3363917538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b200051c0 msgr2=0x7f6b2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b200051c0 0x7f6b2000b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00a850 tx=0x7f6b20005830 comp rx=0 tx=0).stop 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3363917538 shutdown_connections 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3363917538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b200051c0 0x7f6b2000b9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3363917538 >> 172.21.15.50:0/3363917538 conn(0x7f6b3408dc90 msgr2=0x7f6b3409d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3363917538 shutdown_connections 2024-04-02T16:22:08.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.615+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3363917538 wait complete. 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.616+0000 7f6b3b33d640 1 Processor -- start 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.616+0000 7f6b3b33d640 1 -- start start 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b3b33d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200051c0 0x7f6b34160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b3b33d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b340cd740 0x7f6b340cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b3b33d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b340cbd80 0x7f6b3415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b3b33d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340d41b0 con 0x7f6b340cbd80 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b3b33d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340d4030 con 0x7f6b200051c0 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b3b33d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6b340d4330 con 0x7f6b340cd740 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b398da640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b340cbd80 0x7f6b3415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b398da640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b340cbd80 0x7f6b3415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60190/0 (socket says 172.21.15.50:60190) 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b398da640 1 -- 172.21.15.50:0/3459804928 learned_addr learned my addr 172.21.15.50:0/3459804928 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b390d9640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200051c0 0x7f6b34160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.617+0000 7f6b388d8640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b340cd740 0x7f6b340cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b398da640 1 -- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b340cd740 msgr2=0x7f6b340cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b398da640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b340cd740 0x7f6b340cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b398da640 1 -- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200051c0 msgr2=0x7f6b34160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b398da640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200051c0 0x7f6b34160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b398da640 1 -- 172.21.15.50:0/3459804928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b341e8230 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b398da640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b340cbd80 0x7f6b3415c4d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00a850 tx=0x7f6b2c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b29ffb640 1 -- 172.21.15.50:0/3459804928 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c039040 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b29ffb640 1 -- 172.21.15.50:0/3459804928 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c043070 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b34146ba0 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b29ffb640 1 -- 172.21.15.50:0/3459804928 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c004d60 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.618+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b34147160 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.620+0000 7f6b29ffb640 1 -- 172.21.15.50:0/3459804928 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b2c034650 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.620+0000 7f6b29ffb640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6b1806c8f0 0x7f6b1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.620+0000 7f6b29ffb640 1 -- 172.21.15.50:0/3459804928 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6b2c0ada50 con 0x7f6b340cbd80 2024-04-02T16:22:08.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.620+0000 7f6b390d9640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6b1806c8f0 0x7f6b1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.621+0000 7f6b390d9640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6b1806c8f0 0x7f6b1806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000c00 tx=0x7f6b30009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.621+0000 7f6b3dffe740 1 Processor -- start 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.621+0000 7f6b3dffe740 1 -- start start 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781c7ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781caa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781c7b070 0x55c781caae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c781be4060 con 0x55c781ca7b90 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c781b8c650 con 0x55c781c7b070 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c781be0010 con 0x55c781ca7f70 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781c7ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781c7ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60206/0 (socket says 172.21.15.50:60206) 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 -- 172.21.15.50:0/1387822829 learned_addr learned my addr 172.21.15.50:0/1387822829 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b388d8640 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781caa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b398da640 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781c7b070 0x55c781caae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 -- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 msgr2=0x55c781caa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781caa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 -- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781c7b070 msgr2=0x55c781caae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781c7b070 0x55c781caae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 -- 172.21.15.50:0/1387822829 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7818f82d0 con 0x55c781ca7b90 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b390d9640 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781c7ab30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b30012c30 tx=0x7f6b30015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5620feb647e26333 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b1effd640 1 -- 172.21.15.50:0/1387822829 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30019070 con 0x55c781ca7b90 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b1effd640 1 -- 172.21.15.50:0/1387822829 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b3001a040 con 0x55c781ca7b90 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.622+0000 7f6b1effd640 1 -- 172.21.15.50:0/1387822829 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30005020 con 0x55c781ca7b90 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1387822829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 msgr2=0x55c781c7ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781c7ab30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b30012c30 tx=0x7f6b30015000 comp rx=0 tx=0).stop 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1387822829 shutdown_connections 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781c7b070 0x55c781caae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781caa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1387822829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781c7ab30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1387822829 >> 172.21.15.50:0/1387822829 conn(0x55c781c7cda0 msgr2=0x55c781cc5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1387822829 shutdown_connections 2024-04-02T16:22:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.623+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1387822829 wait complete. 2024-04-02T16:22:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.624+0000 7f6b3dffe740 1 -- start start 2024-04-02T16:22:08.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.624+0000 7f6b3dffe740 1 Processor -- start 2024-04-02T16:22:08.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.624+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cc6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781cae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781cb13b0 0x55c781cb1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c781c7f440 con 0x55c781ca7b90 2024-04-02T16:22:08.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c781b9fa10 con 0x55c781cb13b0 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c781c81c80 con 0x55c781ca7f70 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 10 client.0 _ll_register_callbacks cb 0x55c781bdc0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cc6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cc6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60214/0 (socket says 172.21.15.50:60214) 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 -- 172.21.15.50:0/1217473441 learned_addr learned my addr 172.21.15.50:0/1217473441 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b398da640 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781cb13b0 0x55c781cb1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b388d8640 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781cae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 -- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 msgr2=0x55c781cae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781cae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 -- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781cb13b0 msgr2=0x55c781cb1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781cb13b0 0x55c781cb1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 -- 172.21.15.50:0/1217473441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7819c26b0 con 0x55c781ca7b90 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b390d9640 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cc6060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b300216f0 tx=0x7f6b30001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdef19e2416518f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b13fff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b13fff640 1 -- 172.21.15.50:0/1217473441 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3002f070 con 0x55c781ca7b90 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b13fff640 1 -- 172.21.15.50:0/1217473441 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6b30021ae0 con 0x55c781ca7b90 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b3001a040 con 0x55c781ca7b90 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b13fff640 1 -- 172.21.15.50:0/1217473441 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6b3003a050 con 0x55c781ca7b90 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 10 client.4428 fetch_fsmap learned FSMap version 7 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.625+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c781cb07b0 con 0x55c781ca7b90 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b13fff640 1 -- 172.21.15.50:0/1217473441 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b3001a040 con 0x55c781ca7b90 2024-04-02T16:22:08.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b13fff640 1 -- 172.21.15.50:0/1217473441 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f6b3003d020 con 0x55c781ca7b90 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b3dffe740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b3dffe740 10 client.4428 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b3dffe740 20 client.4428 populate_metadata read hostname 'smithi050' 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cd9680 0x55c781ce4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b3dffe740 4 client.4428 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 --> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c781b8bcf0 con 0x55c781cd9680 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b388d8640 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cd9680 0x55c781ce4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b13fff640 10 client.4428 ms_handle_connect on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.626+0000 7f6b388d8640 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cd9680 0x55c781ce4340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d9155792ac0a413 server_cookie=591590f0496858ff in_seq=0 out_seq=0 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b13fff640 1 -- 172.21.15.50:0/1217473441 <== mds.0 v2:172.21.15.188:6834/2647829095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c781b8bcf0 con 0x55c781cd9680 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b13fff640 10 client.4428 handle_command_reply: tid=0 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 client.4428 shutdown 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 msgr2=0x55c781cc6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cc6060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b300216f0 tx=0x7f6b30001120 comp rx=0 tx=0).stop 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 shutdown_connections 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cd9680 0x55c781ce4340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781cb13b0 0x55c781cb1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781ca7f70 0x55c781cae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/1217473441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cc6060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b13fff640 0 client.4428 ms_handle_reset on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 >> 172.21.15.50:0/1217473441 conn(0x55c781c7b070 msgr2=0x55c781c7eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 shutdown_connections 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.627+0000 7f6b3dffe740 1 -- 172.21.15.50:0/1217473441 wait complete. 2024-04-02T16:22:08.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.628+0000 7f6b3dffe740 20 client.4428 trim_cache size 0 max 16384 2024-04-02T16:22:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 Processor -- start 2024-04-02T16:22:08.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 -- start start 2024-04-02T16:22:08.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cafaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7f70 0x55c781ce8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34202690 con 0x55c781ca7b90 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b342016b0 con 0x55c781ca7f70 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.646+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6b34200a40 con 0x55c781c7b070 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cafaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cafaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60226/0 (socket says 172.21.15.50:60226) 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 -- 172.21.15.50:0/258603017 learned_addr learned my addr 172.21.15.50:0/258603017 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b390d9640 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b398da640 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7f70 0x55c781ce8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 -- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 msgr2=0x55c781cd4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 -- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7f70 msgr2=0x55c781ce8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7f70 0x55c781ce8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 -- 172.21.15.50:0/258603017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c781cd5540 con 0x55c781ca7b90 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b390d9640 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b398da640 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7f70 0x55c781ce8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b388d8640 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cafaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b340c8250 tx=0x7f6b24009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0af2f6bf7eb5fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b1e7fc640 1 -- 172.21.15.50:0/258603017 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b240224e0 con 0x55c781ca7b90 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b1e7fc640 1 -- 172.21.15.50:0/258603017 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b24022680 con 0x55c781ca7b90 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.647+0000 7f6b1e7fc640 1 -- 172.21.15.50:0/258603017 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24003840 con 0x55c781ca7b90 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 -- 172.21.15.50:0/258603017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 msgr2=0x55c781cafaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cafaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b340c8250 tx=0x7f6b24009190 comp rx=0 tx=0).stop 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 -- 172.21.15.50:0/258603017 shutdown_connections 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7f70 0x55c781ce8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7b90 0x55c781cafaf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/258603017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 -- 172.21.15.50:0/258603017 >> 172.21.15.50:0/258603017 conn(0x55c781c80e00 msgr2=0x55c781cc60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 -- 172.21.15.50:0/258603017 shutdown_connections 2024-04-02T16:22:08.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 -- 172.21.15.50:0/258603017 wait complete. 2024-04-02T16:22:08.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 -- start start 2024-04-02T16:22:08.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.648+0000 7f6b3dffe740 1 Processor -- start 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7b90 0x55c781cd4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b3dffe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7f70 0x55c781caa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c781cb00e0 con 0x55c781ca7f70 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34201060 con 0x55c781ca7b90 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b3dffe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c781cac170 con 0x55c781c7b070 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b3dffe740 10 client.0 _ll_register_callbacks cb 0x7f6b30012b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b398da640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7f70 0x55c781caa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b398da640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7f70 0x55c781caa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60230/0 (socket says 172.21.15.50:60230) 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b398da640 1 -- 172.21.15.50:0/3491528950 learned_addr learned my addr 172.21.15.50:0/3491528950 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b390d9640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.649+0000 7f6b388d8640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7b90 0x55c781cd4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b398da640 1 -- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 msgr2=0x55c781cd45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b398da640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b398da640 1 -- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7b90 msgr2=0x55c781cd4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b398da640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7b90 0x55c781cd4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b398da640 1 -- 172.21.15.50:0/3491528950 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c781cc8450 con 0x55c781ca7f70 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b388d8640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7b90 0x55c781cd4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b398da640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7f70 0x55c781caa3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c083b80 tx=0x7f6b2c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c55d7231e16658 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b13fff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b13fff640 1 -- 172.21.15.50:0/3491528950 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c098060 con 0x55c781ca7f70 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b13fff640 1 -- 172.21.15.50:0/3491528950 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6b2c077bf0 con 0x55c781ca7f70 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b24022680 con 0x55c781ca7f70 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b13fff640 1 -- 172.21.15.50:0/3491528950 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6b2c0840b0 con 0x55c781ca7f70 2024-04-02T16:22:08.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 10 client.4434 fetch_fsmap learned FSMap version 7 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c781cc7fd0 con 0x55c781ca7f70 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b13fff640 1 -- 172.21.15.50:0/3491528950 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b24022680 con 0x55c781ca7f70 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b13fff640 1 -- 172.21.15.50:0/3491528950 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f6b2c07a9d0 con 0x55c781ca7f70 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 10 client.4434 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 20 client.4434 populate_metadata read hostname 'smithi050' 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cb13b0 0x55c781cc6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 4 client.4434 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 --> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] -- command(tid 0: {"prefix": "status"}) -- 0x55c781bf15a0 con 0x55c781cb13b0 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.650+0000 7f6b390d9640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cb13b0 0x55c781cc6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b13fff640 10 client.4434 ms_handle_connect on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b390d9640 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cb13b0 0x55c781cc6ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c26d25f40fbdf8 server_cookie=91544d61a5f9a34c in_seq=0 out_seq=0 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b13fff640 1 -- 172.21.15.50:0/3491528950 <== mds.0 v2:172.21.15.188:6834/2647829095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c781bf15a0 con 0x55c781cb13b0 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b13fff640 10 client.4434 handle_command_reply: tid=0 2024-04-02T16:22:08.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 client.4434 shutdown 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7f70 msgr2=0x55c781caa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7f70 0x55c781caa3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c083b80 tx=0x7f6b2c0019b0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 shutdown_connections 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x55c781cb13b0 0x55c781cc6ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c781ca7f70 0x55c781caa3f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c781ca7b90 0x55c781cd4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 --2- 172.21.15.50:0/3491528950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c781c7b070 0x55c781cd45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 >> 172.21.15.50:0/3491528950 conn(0x55c781c80e00 msgr2=0x55c781cb1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b13fff640 0 client.4434 ms_handle_reset on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 shutdown_connections 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 1 -- 172.21.15.50:0/3491528950 wait complete. 2024-04-02T16:22:08.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.651+0000 7f6b3dffe740 20 client.4434 trim_cache size 0 max 16384 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "057e6c06-7c98-48a0-9312-9e11e9abe7fe", 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4294, 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 7.2493021999999998, 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 7, 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 25, 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 23, 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 14.411380736 2024-04-02T16:22:08.660 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T16:22:08.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6b1806c8f0 msgr2=0x7f6b1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6b1806c8f0 0x7f6b1806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000c00 tx=0x7f6b30009040 comp rx=0 tx=0).stop 2024-04-02T16:22:08.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b340cbd80 msgr2=0x7f6b3415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b340cbd80 0x7f6b3415c4d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00a850 tx=0x7f6b2c001dc0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 shutdown_connections 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f6b1806c8f0 0x7f6b1806edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b340cbd80 0x7f6b3415c4d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b340cd740 0x7f6b340cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 --2- 172.21.15.50:0/3459804928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b200051c0 0x7f6b34160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 >> 172.21.15.50:0/3459804928 conn(0x7f6b3408dc90 msgr2=0x7f6b34161860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 shutdown_connections 2024-04-02T16:22:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.652+0000 7f6b3b33d640 1 -- 172.21.15.50:0/3459804928 wait complete. 2024-04-02T16:22:08.664 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T16:22:08.664 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:08.664 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:22:08.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.763+0000 7f0718bdb640 1 Processor -- start 2024-04-02T16:22:08.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.763+0000 7f0718bdb640 1 -- start start 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0718bdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0718bdb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f07140b4760 0x7f07140d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0718bdb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f07140d2cb0 0x7f07141622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0718bdb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f07140b5880 con 0x7f07140d2cb0 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0718bdb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f07140b4c70 con 0x7f07140b4760 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0718bdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f07140b4f50 con 0x7f071415e650 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f07140d2cb0 0x7f07141622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f07140d2cb0 0x7f07141622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60238/0 (socket says 172.21.15.50:60238) 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 -- 172.21.15.50:0/986843400 learned_addr learned my addr 172.21.15.50:0/986843400 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f071259c640 1 --2- 172.21.15.50:0/986843400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0711d9b640 1 --2- 172.21.15.50:0/986843400 >> v2:172.21.15.188:3300/0 conn(0x7f07140b4760 0x7f07140d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 -- 172.21.15.50:0/986843400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 msgr2=0x7f07140b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 --2- 172.21.15.50:0/986843400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 -- 172.21.15.50:0/986843400 >> v2:172.21.15.188:3300/0 conn(0x7f07140b4760 msgr2=0x7f07140d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:08.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 --2- 172.21.15.50:0/986843400 >> v2:172.21.15.188:3300/0 conn(0x7f07140b4760 0x7f07140d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0712d9d640 1 -- 172.21.15.50:0/986843400 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07140b5230 con 0x7f07140d2cb0 2024-04-02T16:22:08.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.764+0000 7f0711d9b640 1 --2- 172.21.15.50:0/986843400 >> v2:172.21.15.188:3300/0 conn(0x7f07140b4760 0x7f07140d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:08.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.765+0000 7f0712d9d640 1 --2- 172.21.15.50:0/986843400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07141622e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f071408df00 tx=0x7f070402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99377798ac3e45ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.765+0000 7f071159a640 1 -- 172.21.15.50:0/986843400 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070402ca70 con 0x7f07140d2cb0 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.765+0000 7f071159a640 1 -- 172.21.15.50:0/986843400 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070402cc10 con 0x7f07140d2cb0 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.765+0000 7f071159a640 1 -- 172.21.15.50:0/986843400 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07040343f0 con 0x7f07140d2cb0 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 -- 172.21.15.50:0/986843400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 msgr2=0x7f07141622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 --2- 172.21.15.50:0/986843400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07141622e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f071408df00 tx=0x7f070402bed0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 -- 172.21.15.50:0/986843400 shutdown_connections 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 --2- 172.21.15.50:0/986843400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07141622e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 --2- 172.21.15.50:0/986843400 >> v2:172.21.15.188:3300/0 conn(0x7f07140b4760 0x7f07140d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 --2- 172.21.15.50:0/986843400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 -- 172.21.15.50:0/986843400 >> 172.21.15.50:0/986843400 conn(0x7f071409d580 msgr2=0x7f07140d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 -- 172.21.15.50:0/986843400 shutdown_connections 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.766+0000 7f0718bdb640 1 -- 172.21.15.50:0/986843400 wait complete. 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.767+0000 7f0718bdb640 1 Processor -- start 2024-04-02T16:22:08.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.767+0000 7f0718bdb640 1 -- start start 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0718bdb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07140b4760 0x7f0714150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0718bdb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07140cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0718bdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0718bdb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f071415ea30 con 0x7f07140d2cb0 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0718bdb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07140d2270 con 0x7f07140b4760 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0718bdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f071415ebb0 con 0x7f071415e650 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07140cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07140cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60250/0 (socket says 172.21.15.50:60250) 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 -- 172.21.15.50:0/1983855698 learned_addr learned my addr 172.21.15.50:0/1983855698 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f071259c640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07140b4760 0x7f0714150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0712d9d640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 -- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 msgr2=0x7f07140cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 -- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07140b4760 msgr2=0x7f0714150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07140b4760 0x7f0714150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0711d9b640 1 -- 172.21.15.50:0/1983855698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07140ccc10 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f0712d9d640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.768+0000 7f071259c640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07140b4760 0x7f0714150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.769+0000 7f0711d9b640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07140cbfc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d830 tx=0x7f06fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.769+0000 7f070affd640 1 -- 172.21.15.50:0/1983855698 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc004030 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.769+0000 7f070affd640 1 -- 172.21.15.50:0/1983855698 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc0041d0 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.769+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07140ccef0 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.769+0000 7f070affd640 1 -- 172.21.15.50:0/1983855698 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00b6c0 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.769+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07140cd4b0 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.770+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07140756f0 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.770+0000 7f070affd640 1 -- 172.21.15.50:0/1983855698 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06fc00ba70 con 0x7f07140d2cb0 2024-04-02T16:22:08.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.771+0000 7f070affd640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f06f406c8f0 0x7f06f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:08.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.771+0000 7f070affd640 1 -- 172.21.15.50:0/1983855698 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f06fc08c360 con 0x7f07140d2cb0 2024-04-02T16:22:08.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.771+0000 7f071259c640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f06f406c8f0 0x7f06f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:08.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.771+0000 7f071259c640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f06f406c8f0 0x7f06f406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:08.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.775+0000 7f070affd640 1 -- 172.21.15.50:0/1983855698 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f06fc05a450 con 0x7f07140d2cb0 2024-04-02T16:22:08.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.996+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f07140d1aa0 con 0x7f07140d2cb0 2024-04-02T16:22:08.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.997+0000 7f070affd640 1 -- 172.21.15.50:0/1983855698 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f06fc059d50 con 0x7f07140d2cb0 2024-04-02T16:22:08.998 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T16:22:09.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f06f406c8f0 msgr2=0x7f06f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:09.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f06f406c8f0 0x7f06f406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f070c006fd0 tx=0x7f070c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 msgr2=0x7f07140cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07140cbfc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d830 tx=0x7f06fc00dcf0 comp rx=0 tx=0).stop 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 shutdown_connections 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071415e650 0x7f07140cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f06f406c8f0 0x7f06f406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07140d2cb0 0x7f07140cbfc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 --2- 172.21.15.50:0/1983855698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07140b4760 0x7f0714150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 >> 172.21.15.50:0/1983855698 conn(0x7f071409d580 msgr2=0x7f07140b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 shutdown_connections 2024-04-02T16:22:09.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:08.999+0000 7f0718bdb640 1 -- 172.21.15.50:0/1983855698 wait complete. 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:22:09.011 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T16:22:09.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:09.012 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42721sec preferred_lft 42721sec 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-04-02T16:22:09.029 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T16:22:09.029 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:09.029 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:09.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-04-02T16:22:09.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-04-02T16:22:09.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-04-02T16:22:09.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T16:22:09.029 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:09.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:09.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:09.222 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:22:09.249 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-04-02T16:22:09.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:09.251 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-04-02T16:22:09.316 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-04-02T16:22:09.316 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:22:09.316 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T16:22:09.316 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T16:22:09.316 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:22:09.317 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:09.317 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:09.317 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T16:22:09.317 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T16:22:09.317 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T16:22:09.317 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:09.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:09.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:09.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:09.673 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-04-02T16:22:09.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:09.689 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-04-02T16:22:09.744 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:09.744 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:09.744 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T16:22:09.744 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-02T16:22:09.744 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:09.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:09.865 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-02T16:22:09.866 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:09.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:09.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:10.003 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:10.003 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:10.003 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-04-02T16:22:10.004 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T16:22:10.004 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:10.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:10.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:10.084 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-02T16:22:10.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:10.084 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:10.098 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T16:22:10.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:10.098 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:10.153 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-04-02T16:22:10.220 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:22:10.276 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-04-02T16:22:10.276 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-04-02T16:22:10.276 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T16:22:10.277 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T16:22:10.278 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T16:22:10.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:10.279 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-04-02T16:22:10.331 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T16:22:10.331 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-02T16:22:10.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-04-02T16:22:10.400 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:22:10.457 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-04-02T16:22:10.458 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T16:22:10.459 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T16:22:10.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:10.460 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-04-02T16:22:10.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:2024-04-02T16:22:10.462+0000 7ffbf7983480 -1 init, newargv = 0x5580cfa8c320 newargc=15 2024-04-02T16:22:10.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[44150]: starting ceph client 2024-04-02T16:22:10.469 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[44150]: starting fuse 2024-04-02T16:22:10.477 INFO:teuthology.orchestra.run.smithi050.stdout:42 2024-04-02T16:22:10.477 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-02T16:22:10.477 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2024-04-02T16:22:10.477 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2024-04-02T16:22:10.477 DEBUG:teuthology.orchestra.run.smithi050:> import re 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> import os 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T16:22:10.478 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> return f 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.admin")) 2024-04-02T16:22:10.479 DEBUG:teuthology.orchestra.run.smithi050:> ' 2024-04-02T16:22:10.597 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.admin.44150.asok 2024-04-02T16:22:10.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:10.608 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44150.asok 2024-04-02T16:22:10.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:10.608 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44150.asok status 2024-04-02T16:22:10.745 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T16:22:10.745 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "191d963ac9de7322da8aff20b567b3fa458e2ef6", 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "admin", 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "44150", 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4446, 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2024-04-02T16:22:10.746 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 4446 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1274707859 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1274707859 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.4446 192.168.0.1:0/1274707859", 2024-04-02T16:22:10.747 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/1274707859", 2024-04-02T16:22:10.748 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2024-04-02T16:22:10.748 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 7, 2024-04-02T16:22:10.748 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 25, 2024-04-02T16:22:10.748 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2024-04-02T16:22:10.748 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2024-04-02T16:22:10.748 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2024-04-02T16:22:10.748 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T16:22:10.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:10.753 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2024-04-02T16:22:10.753 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:10.768 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2024-04-02T16:22:10.769 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:10.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:10.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:10.837 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-02T16:22:10.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:10.944 INFO:teuthology.orchestra.run.smithi050.stdout:# file: . 2024-04-02T16:22:10.945 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-02T16:22:10.945 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:10.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:10.950 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:10.950 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:10.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:11.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.106+0000 7ffb9c1e9640 1 Processor -- start 2024-04-02T16:22:11.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.106+0000 7ffb9c1e9640 1 -- start start 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb9c1e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 0x7ffb940d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb9c1e9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9406f840 con 0x7ffb9409aa40 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb9c1e9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9406fb20 con 0x7ffb940b45b0 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb9c1e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffb940cc5f0 con 0x7ffb940ce0e0 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb99f85640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ffb9409aa40 0x7ffb9409ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49476/0 (socket says 172.21.15.50:49476) 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb99f85640 1 -- 172.21.15.50:0/3028815605 learned_addr learned my addr 172.21.15.50:0/3028815605 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb9a786640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 0x7ffb940d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 672754297 0 0) 0x7ffb9406f840 con 0x7ffb9409aa40 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.107+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb7c003520 con 0x7ffb9409aa40 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1059427628 0 0) 0x7ffb7c003520 con 0x7ffb9409aa40 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 msgr2=0x7ffb940d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 0x7ffb940d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 >> v1:172.21.15.188:6789/0 conn(0x7ffb940b45b0 legacy=0x7ffb940d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb940cc8d0 con 0x7ffb9409aa40 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7ffb8c001670 con 0x7ffb9409aa40 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 >> v1:172.21.15.50:6789/0 conn(0x7ffb9409aa40 legacy=0x7ffb9409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb7c0039e0 0x7ffb7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c004800 0x7ffb7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 0x7ffb7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb940cc5f0 con 0x7ffb7c0039e0 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c00baa0 con 0x7ffb7c004800 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7c00bca0 con 0x7ffb940ce0e0 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb99f85640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb7c0039e0 0x7ffb7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb99784640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c004800 0x7ffb7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb9a786640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 0x7ffb7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb99f85640 1 -- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 msgr2=0x7ffb7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb99f85640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 0x7ffb7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb99f85640 1 -- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c004800 msgr2=0x7ffb7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb99f85640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c004800 0x7ffb7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb99f85640 1 -- 172.21.15.50:0/3028815605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c003520 con 0x7ffb7c0039e0 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.108+0000 7ffb9a786640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940ce0e0 0x7ffb7c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb99f85640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb7c0039e0 0x7ffb7c003e00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c001050 tx=0x7ffb7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecae337b8f0138 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c007d30 con 0x7ffb7c0039e0 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c030070 con 0x7ffb7c0039e0 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb98f83640 1 -- 172.21.15.50:0/3028815605 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c005d30 con 0x7ffb7c0039e0 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/3028815605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb7c0039e0 msgr2=0x7ffb7c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb9c1e9640 1 --2- 172.21.15.50:0/3028815605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb7c0039e0 0x7ffb7c003e00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c001050 tx=0x7ffb7c005830 comp rx=0 tx=0).stop 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb9a786640 1 -- 172.21.15.50:0/3028815605 reap_dead start 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/3028815605 shutdown_connections 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.109+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/3028815605 >> 172.21.15.50:0/3028815605 conn(0x7ffb940b3a90 msgr2=0x7ffb940b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.110+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/3028815605 shutdown_connections 2024-04-02T16:22:11.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.110+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/3028815605 wait complete. 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 Processor -- start 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 -- start start 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c0039e0 0x7ffb94152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb9406d490 0x7ffb9406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb941f7940 0x7ffb941f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9409b9e0 con 0x7ffb9406d490 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9409b860 con 0x7ffb7c0039e0 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.111+0000 7ffb9c1e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9415e890 con 0x7ffb941f7940 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb9406d490 0x7ffb9406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb9406d490 0x7ffb9406d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60268/0 (socket says 172.21.15.50:60268) 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 -- 172.21.15.50:0/1757830321 learned_addr learned my addr 172.21.15.50:0/1757830321 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99f85640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c0039e0 0x7ffb94152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb9a786640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb941f7940 0x7ffb941f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 -- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb941f7940 msgr2=0x7ffb941f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb941f7940 0x7ffb941f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 -- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c0039e0 msgr2=0x7ffb94152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c0039e0 0x7ffb94152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 -- 172.21.15.50:0/1757830321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb940ca0c0 con 0x7ffb9406d490 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99f85640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c0039e0 0x7ffb94152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb9a786640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb941f7940 0x7ffb941f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:11.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb99784640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb9406d490 0x7ffb9406d8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800f620 tx=0x7ffb8800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb927fc640 1 -- 172.21.15.50:0/1757830321 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880052e0 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb927fc640 1 -- 172.21.15.50:0/1757830321 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb88018070 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb940ca3a0 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb927fc640 1 -- 172.21.15.50:0/1757830321 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88003810 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.112+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb940ca960 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.113+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb94070060 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.114+0000 7ffb927fc640 1 -- 172.21.15.50:0/1757830321 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb880039b0 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.114+0000 7ffb927fc640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8006c8f0 0x7ffb8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.114+0000 7ffb927fc640 1 -- 172.21.15.50:0/1757830321 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffb8808bc50 con 0x7ffb9406d490 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.114+0000 7ffb99f85640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8006c8f0 0x7ffb8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.115+0000 7ffb99f85640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8006c8f0 0x7ffb8006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c034000 tx=0x7ffb8c007000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.119+0000 7ffb927fc640 1 -- 172.21.15.50:0/1757830321 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffb94070060 con 0x7ffb9406d490 2024-04-02T16:22:11.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.303+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb940756f0 con 0x7ffb9406d490 2024-04-02T16:22:11.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.304+0000 7ffb927fc640 1 -- 172.21.15.50:0/1757830321 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7ffb88091020 con 0x7ffb9406d490 2024-04-02T16:22:11.306 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:11.306 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8006c8f0 msgr2=0x7ffb8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8006c8f0 0x7ffb8006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c034000 tx=0x7ffb8c007000 comp rx=0 tx=0).stop 2024-04-02T16:22:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb9406d490 msgr2=0x7ffb9406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb9406d490 0x7ffb9406d8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800f620 tx=0x7ffb8800fae0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 shutdown_connections 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb941f7940 0x7ffb941f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb8006c8f0 0x7ffb8006edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb9406d490 0x7ffb9406d8f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 --2- 172.21.15.50:0/1757830321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb7c0039e0 0x7ffb94152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 >> 172.21.15.50:0/1757830321 conn(0x7ffb940b3a90 msgr2=0x7ffb940b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 shutdown_connections 2024-04-02T16:22:11.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.307+0000 7ffb9c1e9640 1 -- 172.21.15.50:0/1757830321 wait complete. 2024-04-02T16:22:11.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:11.320 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:11.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.422+0000 7f79fa174640 1 Processor -- start 2024-04-02T16:22:11.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.422+0000 7f79fa174640 1 -- start start 2024-04-02T16:22:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79fa174640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162600 0x7f79f40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79fa174640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f79f41629e0 0x7f79f40cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79fa174640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f409b450 con 0x7f79f4162f30 2024-04-02T16:22:11.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79fa174640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f79f40d0aa0 con 0x7f79f41629e0 2024-04-02T16:22:11.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79fa174640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f79f409b6f0 con 0x7f79f4162600 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79f3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f79f41629e0 0x7f79f40cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79f2ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f79f4162f30 0x7f79f40d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49482/0 (socket says 172.21.15.50:49482) 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79f3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f79f41629e0 0x7f79f40cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60270/0 (socket says 172.21.15.50:60270) 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79f2ffd640 1 -- 172.21.15.50:0/342693247 learned_addr learned my addr 172.21.15.50:0/342693247 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.423+0000 7f79f37fe640 1 --2- 172.21.15.50:0/342693247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162600 0x7f79f40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.424+0000 7f79f3fff640 1 -- 172.21.15.50:0/342693247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162600 msgr2=0x7f79f40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.424+0000 7f79f3fff640 1 --2- 172.21.15.50:0/342693247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162600 0x7f79f40b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.424+0000 7f79f3fff640 1 -- 172.21.15.50:0/342693247 >> v1:172.21.15.50:6789/0 conn(0x7f79f4162f30 legacy=0x7f79f40d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.424+0000 7f79f27fc640 1 -- 172.21.15.50:0/342693247 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3043776551 0 0) 0x7f79f409b450 con 0x7f79f4162f30 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.424+0000 7f79f3fff640 1 -- 172.21.15.50:0/342693247 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f409b9d0 con 0x7f79f41629e0 2024-04-02T16:22:11.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.424+0000 7f79f3fff640 1 --2- 172.21.15.50:0/342693247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f40cc780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009d90 tx=0x7f79ec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58433c656b58c6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.424+0000 7f79f27fc640 1 -- 172.21.15.50:0/342693247 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec039070 con 0x7f79f41629e0 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79f27fc640 1 -- 172.21.15.50:0/342693247 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec02cc50 con 0x7f79f41629e0 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79f27fc640 1 -- 172.21.15.50:0/342693247 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec034440 con 0x7f79f41629e0 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 -- 172.21.15.50:0/342693247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 msgr2=0x7f79f40cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 --2- 172.21.15.50:0/342693247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f40cc780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009d90 tx=0x7f79ec02bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 -- 172.21.15.50:0/342693247 shutdown_connections 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 --2- 172.21.15.50:0/342693247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f40cc780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 --2- 172.21.15.50:0/342693247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162600 0x7f79f40b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 -- 172.21.15.50:0/342693247 >> 172.21.15.50:0/342693247 conn(0x7f79f40b3c20 msgr2=0x7f79f40b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 -- 172.21.15.50:0/342693247 shutdown_connections 2024-04-02T16:22:11.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.425+0000 7f79fa174640 1 -- 172.21.15.50:0/342693247 wait complete. 2024-04-02T16:22:11.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.426+0000 7f79fa174640 1 Processor -- start 2024-04-02T16:22:11.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.426+0000 7f79fa174640 1 -- start start 2024-04-02T16:22:11.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79fa174640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f4162600 0x7f79f40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79fa174640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f4152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79fa174640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162f30 0x7f79f4150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79fa174640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79f40d0350 con 0x7f79f41629e0 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79fa174640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79f40d1640 con 0x7f79f4162600 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79fa174640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f79f40d04d0 con 0x7f79f4162f30 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f4152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f4152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60274/0 (socket says 172.21.15.50:60274) 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 -- 172.21.15.50:0/1479811688 learned_addr learned my addr 172.21.15.50:0/1479811688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f3fff640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162f30 0x7f79f4150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f37fe640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f4162600 0x7f79f40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 -- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162f30 msgr2=0x7f79f4150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162f30 0x7f79f4150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 -- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f4162600 msgr2=0x7f79f40cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f4162600 0x7f79f40cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.427+0000 7f79f2ffd640 1 -- 172.21.15.50:0/1479811688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f40af430 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.428+0000 7f79f2ffd640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f4152820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c430 tx=0x7f79e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.428+0000 7f79dbfff640 1 -- 172.21.15.50:0/1479811688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e80074e0 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.428+0000 7f79dbfff640 1 -- 172.21.15.50:0/1479811688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e80047e0 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.428+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f40af710 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.428+0000 7f79dbfff640 1 -- 172.21.15.50:0/1479811688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8004bd0 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.428+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f40b0ed0 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.429+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f4075760 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.429+0000 7f79dbfff640 1 -- 172.21.15.50:0/1479811688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79e8011070 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.430+0000 7f79dbfff640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f79e006c8f0 0x7f79e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.430+0000 7f79dbfff640 1 -- 172.21.15.50:0/1479811688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f79e808ba30 con 0x7f79f41629e0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.430+0000 7f79f37fe640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f79e006c8f0 0x7f79e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.430+0000 7f79f37fe640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f79e006c8f0 0x7f79e006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79e4006fd0 tx=0x7f79e4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.434+0000 7f79dbfff640 1 -- 172.21.15.50:0/1479811688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f79e800e280 con 0x7f79f41629e0 2024-04-02T16:22:11.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.620+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79f40b7eb0 con 0x7f79f41629e0 2024-04-02T16:22:11.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.620+0000 7f79dbfff640 1 -- 172.21.15.50:0/1479811688 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f79e8059bb0 con 0x7f79f41629e0 2024-04-02T16:22:11.622 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.622+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f79e006c8f0 msgr2=0x7f79e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.622+0000 7f79fa174640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f79e006c8f0 0x7f79e006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79e4006fd0 tx=0x7f79e4008040 comp rx=0 tx=0).stop 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.622+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 msgr2=0x7f79f4152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.622+0000 7f79fa174640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f4152820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c430 tx=0x7f79e800c8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.622+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 shutdown_connections 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.623+0000 7f79fa174640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f4162f30 0x7f79f4150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.623+0000 7f79fa174640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f79e006c8f0 0x7f79e006edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.623+0000 7f79fa174640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79f41629e0 0x7f79f4152820 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.623+0000 7f79fa174640 1 --2- 172.21.15.50:0/1479811688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f4162600 0x7f79f40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.623+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 >> 172.21.15.50:0/1479811688 conn(0x7f79f40b3c20 msgr2=0x7f79f406ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:11.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.623+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 shutdown_connections 2024-04-02T16:22:11.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.623+0000 7f79fa174640 1 -- 172.21.15.50:0/1479811688 wait complete. 2024-04-02T16:22:11.635 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.738+0000 7f8baa704640 1 Processor -- start 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.738+0000 7f8baa704640 1 -- start start 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.738+0000 7f8baa704640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ba415e3d0 0x7f8ba40d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.738+0000 7f8baa704640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e7b0 0x7f8ba40d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.738+0000 7f8baa704640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba40d2f30 con 0x7f8ba40c7e30 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.738+0000 7f8baa704640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba40cf000 con 0x7f8ba415e3d0 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.738+0000 7f8baa704640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba40cf2e0 con 0x7f8ba415e7b0 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8ba415e3d0 0x7f8ba40d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba37fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f8ba40c7e30 0x7f8ba40d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49492/0 (socket says 172.21.15.50:49492) 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba37fe640 1 -- 172.21.15.50:0/3491549175 learned_addr learned my addr 172.21.15.50:0/3491549175 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba8ca1640 1 --2- 172.21.15.50:0/3491549175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e7b0 0x7f8ba40d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba3fff640 1 -- 172.21.15.50:0/3491549175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e7b0 msgr2=0x7f8ba40d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba3fff640 1 --2- 172.21.15.50:0/3491549175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e7b0 0x7f8ba40d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba3fff640 1 -- 172.21.15.50:0/3491549175 >> v1:172.21.15.50:6789/0 conn(0x7f8ba40c7e30 legacy=0x7f8ba40d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba2ffd640 1 -- 172.21.15.50:0/3491549175 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117741256 0 0) 0x7f8ba40d2f30 con 0x7f8ba40c7e30 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.739+0000 7f8ba3fff640 1 -- 172.21.15.50:0/3491549175 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba40cf5c0 con 0x7f8ba415e3d0 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8ba3fff640 1 --2- 172.21.15.50:0/3491549175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e3d0 0x7f8ba40d4030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0097a0 tx=0x7f8b8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930c61f9d7b9a2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8ba2ffd640 1 -- 172.21.15.50:0/3491549175 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c035020 con 0x7f8ba415e3d0 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8ba2ffd640 1 -- 172.21.15.50:0/3491549175 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c031830 con 0x7f8ba415e3d0 2024-04-02T16:22:11.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8ba2ffd640 1 -- 172.21.15.50:0/3491549175 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c031c00 con 0x7f8ba415e3d0 2024-04-02T16:22:11.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 -- 172.21.15.50:0/3491549175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e3d0 msgr2=0x7f8ba40d4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 --2- 172.21.15.50:0/3491549175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e3d0 0x7f8ba40d4030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0097a0 tx=0x7f8b8c02b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 -- 172.21.15.50:0/3491549175 shutdown_connections 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 --2- 172.21.15.50:0/3491549175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e7b0 0x7f8ba40d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 --2- 172.21.15.50:0/3491549175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e3d0 0x7f8ba40d4030 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 -- 172.21.15.50:0/3491549175 >> 172.21.15.50:0/3491549175 conn(0x7f8ba408dc90 msgr2=0x7f8ba409d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 -- 172.21.15.50:0/3491549175 shutdown_connections 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.740+0000 7f8baa704640 1 -- 172.21.15.50:0/3491549175 wait complete. 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.742+0000 7f8baa704640 1 Processor -- start 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.742+0000 7f8baa704640 1 -- start start 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.742+0000 7f8baa704640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba40c7e30 0x7f8ba4160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.742+0000 7f8baa704640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e3d0 0x7f8ba40b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.742+0000 7f8baa704640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e7b0 0x7f8ba409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.742+0000 7f8baa704640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40d46f0 con 0x7f8ba415e7b0 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.742+0000 7f8baa704640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40d4570 con 0x7f8ba40c7e30 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8baa704640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba40d4870 con 0x7f8ba415e3d0 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e7b0 0x7f8ba409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e7b0 0x7f8ba409ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60302/0 (socket says 172.21.15.50:60302) 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 -- 172.21.15.50:0/2247213227 learned_addr learned my addr 172.21.15.50:0/2247213227 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba37fe640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e3d0 0x7f8ba40b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba3fff640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba40c7e30 0x7f8ba4160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 -- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e3d0 msgr2=0x7f8ba40b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e3d0 0x7f8ba40b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 -- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba40c7e30 msgr2=0x7f8ba4160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba40c7e30 0x7f8ba4160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 -- 172.21.15.50:0/2247213227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba415edf0 con 0x7f8ba415e7b0 2024-04-02T16:22:11.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba3fff640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba40c7e30 0x7f8ba4160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba8ca1640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e7b0 0x7f8ba409ed80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00efc0 tx=0x7f8b9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba0ff9640 1 -- 172.21.15.50:0/2247213227 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c018020 con 0x7f8ba415e7b0 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba0ff9640 1 -- 172.21.15.50:0/2247213227 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c014480 con 0x7f8ba415e7b0 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba415f0d0 con 0x7f8ba415e7b0 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8ba0ff9640 1 -- 172.21.15.50:0/2247213227 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c014870 con 0x7f8ba415e7b0 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.743+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba415f690 con 0x7f8ba415e7b0 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.744+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba4073110 con 0x7f8ba415e7b0 2024-04-02T16:22:11.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.745+0000 7f8ba0ff9640 1 -- 172.21.15.50:0/2247213227 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b9c005d10 con 0x7f8ba415e7b0 2024-04-02T16:22:11.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.745+0000 7f8ba0ff9640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b8406c8f0 0x7f8b8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:11.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.745+0000 7f8ba0ff9640 1 -- 172.21.15.50:0/2247213227 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8b9c08cd60 con 0x7f8ba415e7b0 2024-04-02T16:22:11.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.745+0000 7f8ba3fff640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b8406c8f0 0x7f8b8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:11.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.746+0000 7f8ba3fff640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b8406c8f0 0x7f8b8406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c036000 tx=0x7f8b8c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:11.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.749+0000 7f8ba0ff9640 1 -- 172.21.15.50:0/2247213227 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8ba4073110 con 0x7f8ba415e7b0 2024-04-02T16:22:11.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.932+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ba40cb170 con 0x7f8ba415e7b0 2024-04-02T16:22:11.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.932+0000 7f8ba0ff9640 1 -- 172.21.15.50:0/2247213227 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f8b9c022030 con 0x7f8ba415e7b0 2024-04-02T16:22:11.934 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b8406c8f0 msgr2=0x7f8b8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b8406c8f0 0x7f8b8406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c036000 tx=0x7f8b8c008030 comp rx=0 tx=0).stop 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e7b0 msgr2=0x7f8ba409ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e7b0 0x7f8ba409ed80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00efc0 tx=0x7f8b9c00c490 comp rx=0 tx=0).stop 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 shutdown_connections 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b8406c8f0 0x7f8b8406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ba415e7b0 0x7f8ba409ed80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba415e3d0 0x7f8ba40b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 --2- 172.21.15.50:0/2247213227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba40c7e30 0x7f8ba4160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:11.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 >> 172.21.15.50:0/2247213227 conn(0x7f8ba408dc90 msgr2=0x7f8ba40d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:11.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 shutdown_connections 2024-04-02T16:22:11.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:11.934+0000 7f8baa704640 1 -- 172.21.15.50:0/2247213227 wait complete. 2024-04-02T16:22:11.946 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:12.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.049+0000 7fc89376d640 1 Processor -- start 2024-04-02T16:22:12.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.049+0000 7fc89376d640 1 -- start start 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc89376d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc88c0d0b40 0x7fc88c0d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc89376d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc88c06cc40 con 0x7fc88c09b150 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc89376d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc88c06c960 con 0x7fc88c071020 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc89376d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fc88c0ce4a0 con 0x7fc88c0d0b40 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc891d0a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc88c0d0b40 0x7fc88c0d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc890d08640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc88c09b150 0x7fc88c0d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49502/0 (socket says 172.21.15.50:49502) 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc890d08640 1 -- 172.21.15.50:0/3806145631 learned_addr learned my addr 172.21.15.50:0/3806145631 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc891d0a640 1 -- 172.21.15.50:0/3806145631 >> v1:172.21.15.188:6789/0 conn(0x7fc88c071020 legacy=0x7fc88c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc891d0a640 1 -- 172.21.15.50:0/3806145631 >> v1:172.21.15.50:6789/0 conn(0x7fc88c09b150 legacy=0x7fc88c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.050+0000 7fc891d0a640 1 -- 172.21.15.50:0/3806145631 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c0cc6d0 con 0x7fc88c0d0b40 2024-04-02T16:22:12.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.051+0000 7fc891d0a640 1 --2- 172.21.15.50:0/3806145631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0d0ff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009d90 tx=0x7fc87c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb39a897401d6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.051+0000 7fc88bfff640 1 -- 172.21.15.50:0/3806145631 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c039070 con 0x7fc88c0d0b40 2024-04-02T16:22:12.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.051+0000 7fc88bfff640 1 -- 172.21.15.50:0/3806145631 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87c02cd00 con 0x7fc88c0d0b40 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.051+0000 7fc88bfff640 1 -- 172.21.15.50:0/3806145631 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c034660 con 0x7fc88c0d0b40 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.052+0000 7fc89376d640 1 -- 172.21.15.50:0/3806145631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c0d0b40 msgr2=0x7fc88c0d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.052+0000 7fc89376d640 1 --2- 172.21.15.50:0/3806145631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0d0ff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009d90 tx=0x7fc87c02bf50 comp rx=0 tx=0).stop 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.052+0000 7fc89376d640 1 -- 172.21.15.50:0/3806145631 shutdown_connections 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.052+0000 7fc89376d640 1 --2- 172.21.15.50:0/3806145631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0d0ff0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.052+0000 7fc89376d640 1 -- 172.21.15.50:0/3806145631 >> 172.21.15.50:0/3806145631 conn(0x7fc88c0b3ad0 msgr2=0x7fc88c0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.052+0000 7fc89376d640 1 -- 172.21.15.50:0/3806145631 shutdown_connections 2024-04-02T16:22:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.052+0000 7fc89376d640 1 -- 172.21.15.50:0/3806145631 wait complete. 2024-04-02T16:22:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.053+0000 7fc89376d640 1 Processor -- start 2024-04-02T16:22:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.053+0000 7fc89376d640 1 -- start start 2024-04-02T16:22:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc89376d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc88c071020 0x7fc88c0c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc89376d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c09b150 0x7fc88c0c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc89376d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc89376d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c0ccf70 con 0x7fc88c09b150 2024-04-02T16:22:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc89376d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c0ccc90 con 0x7fc88c0d0b40 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc89376d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc88c06db50 con 0x7fc88c071020 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c09b150 0x7fc88c0c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c09b150 0x7fc88c0c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60314/0 (socket says 172.21.15.50:60314) 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 -- 172.21.15.50:0/1740646662 learned_addr learned my addr 172.21.15.50:0/1740646662 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 -- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc88c071020 msgr2=0x7fc88c0c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc88c071020 0x7fc88c0c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 -- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c0d0b40 msgr2=0x7fc88c0c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc891d0a640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 -- 172.21.15.50:0/1740646662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c1e0e40 con 0x7fc88c09b150 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc891d0a640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.054+0000 7fc890d08640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c09b150 0x7fc88c0c5b10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc88000c400 tx=0x7fc88000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.055+0000 7fc889ffb640 1 -- 172.21.15.50:0/1740646662 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880017070 con 0x7fc88c09b150 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.055+0000 7fc889ffb640 1 -- 172.21.15.50:0/1740646662 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc880005ce0 con 0x7fc88c09b150 2024-04-02T16:22:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.055+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c1e1120 con 0x7fc88c09b150 2024-04-02T16:22:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.055+0000 7fc889ffb640 1 -- 172.21.15.50:0/1740646662 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8800072f0 con 0x7fc88c09b150 2024-04-02T16:22:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.055+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c0c7ce0 con 0x7fc88c09b150 2024-04-02T16:22:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.056+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c075760 con 0x7fc88c09b150 2024-04-02T16:22:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.056+0000 7fc889ffb640 1 -- 172.21.15.50:0/1740646662 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc88001a070 con 0x7fc88c09b150 2024-04-02T16:22:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.056+0000 7fc889ffb640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc87806c8f0 0x7fc87806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.056+0000 7fc889ffb640 1 -- 172.21.15.50:0/1740646662 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc88008b2a0 con 0x7fc88c09b150 2024-04-02T16:22:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.057+0000 7fc891509640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc87806c8f0 0x7fc87806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.057+0000 7fc891509640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc87806c8f0 0x7fc87806edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc884006fd0 tx=0x7fc884006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.061+0000 7fc889ffb640 1 -- 172.21.15.50:0/1740646662 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc880059390 con 0x7fc88c09b150 2024-04-02T16:22:12.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.247+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc88c1e9350 con 0x7fc88c09b150 2024-04-02T16:22:12.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.247+0000 7fc889ffb640 1 -- 172.21.15.50:0/1740646662 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc880058c90 con 0x7fc88c09b150 2024-04-02T16:22:12.248 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc87806c8f0 msgr2=0x7fc87806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc87806c8f0 0x7fc87806edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc884006fd0 tx=0x7fc884006bc0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c09b150 msgr2=0x7fc88c0c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c09b150 0x7fc88c0c5b10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc88000c400 tx=0x7fc88000c8c0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 shutdown_connections 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c0d0b40 0x7fc88c0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc87806c8f0 0x7fc87806edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc88c09b150 0x7fc88c0c5b10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 --2- 172.21.15.50:0/1740646662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc88c071020 0x7fc88c0c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 >> 172.21.15.50:0/1740646662 conn(0x7fc88c0b3ad0 msgr2=0x7fc88c1e56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 shutdown_connections 2024-04-02T16:22:12.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.249+0000 7fc89376d640 1 -- 172.21.15.50:0/1740646662 wait complete. 2024-04-02T16:22:12.261 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:12.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.365+0000 7ff0df814640 1 Processor -- start 2024-04-02T16:22:12.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.366+0000 7ff0df814640 1 -- start start 2024-04-02T16:22:12.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.366+0000 7ff0df814640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 0x7ff0d80d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.366+0000 7ff0df814640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d80b10e0 con 0x7ff0d80d2e50 2024-04-02T16:22:12.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.366+0000 7ff0df814640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d80b1380 con 0x7ff0d80d13d0 2024-04-02T16:22:12.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.366+0000 7ff0df814640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d80b1660 con 0x7ff0d8161c80 2024-04-02T16:22:12.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.366+0000 7ff0dcdaf640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff0d80d2e50 0x7ff0d80d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49512/0 (socket says 172.21.15.50:49512) 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.366+0000 7ff0dcdaf640 1 -- 172.21.15.50:0/618568099 learned_addr learned my addr 172.21.15.50:0/618568099 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dd5b0640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 0x7ff0d80d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819070158 0 0) 0x7ff0d80b10e0 con 0x7ff0d80d2e50 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0c4003520 con 0x7ff0d80d2e50 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3940723707 0 0) 0x7ff0c4003520 con 0x7ff0d80d2e50 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 msgr2=0x7ff0d80d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 0x7ff0d80d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 >> v1:172.21.15.188:6789/0 conn(0x7ff0d80d13d0 legacy=0x7ff0d80b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d80d3340 con 0x7ff0d80d2e50 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7ff0c8001670 con 0x7ff0d80d2e50 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 >> v1:172.21.15.50:6789/0 conn(0x7ff0d80d2e50 legacy=0x7ff0d80d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 0x7ff0c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c4003f60 0x7ff0c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d80b1660 con 0x7ff0c4004d40 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c400bb50 con 0x7ff0c4003f60 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c400bd50 con 0x7ff0d8161c80 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dddb1640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dd5b0640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 0x7ff0c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dcdaf640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c4003f60 0x7ff0c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dddb1640 1 -- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 msgr2=0x7ff0c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dddb1640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d8161c80 0x7ff0c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dddb1640 1 -- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c4003f60 msgr2=0x7ff0c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dddb1640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0c4003f60 0x7ff0c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.367+0000 7ff0dddb1640 1 -- 172.21.15.50:0/618568099 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c4003520 con 0x7ff0c4004d40 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.368+0000 7ff0dddb1640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0c40051c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0001e50 tx=0x7ff0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3b9ef7a197a8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.368+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0001a70 con 0x7ff0c4004d40 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.368+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d0003b70 con 0x7ff0c4004d40 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.368+0000 7ff0cffff640 1 -- 172.21.15.50:0/618568099 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00050d0 con 0x7ff0c4004d40 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.369+0000 7ff0df814640 1 -- 172.21.15.50:0/618568099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 msgr2=0x7ff0c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.369+0000 7ff0df814640 1 --2- 172.21.15.50:0/618568099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0c40051c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0001e50 tx=0x7ff0c4005830 comp rx=0 tx=0).stop 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.369+0000 7ff0dddb1640 1 -- 172.21.15.50:0/618568099 reap_dead start 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.369+0000 7ff0df814640 1 -- 172.21.15.50:0/618568099 shutdown_connections 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.369+0000 7ff0df814640 1 -- 172.21.15.50:0/618568099 >> 172.21.15.50:0/618568099 conn(0x7ff0d809d5e0 msgr2=0x7ff0d80d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.369+0000 7ff0df814640 1 -- 172.21.15.50:0/618568099 shutdown_connections 2024-04-02T16:22:12.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.369+0000 7ff0df814640 1 -- 172.21.15.50:0/618568099 wait complete. 2024-04-02T16:22:12.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.370+0000 7ff0df814640 1 Processor -- start 2024-04-02T16:22:12.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.370+0000 7ff0df814640 1 -- start start 2024-04-02T16:22:12.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0df814640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0d80ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0df814640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d80ca640 0x7ff0d815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0df814640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d815b3c0 0x7ff0d8147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0df814640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d80d2910 con 0x7ff0c4004d40 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0df814640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d80d0390 con 0x7ff0d815b3c0 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0df814640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d80d2a90 con 0x7ff0d80ca640 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0d80ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0d80ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60330/0 (socket says 172.21.15.50:60330) 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 -- 172.21.15.50:0/2503943827 learned_addr learned my addr 172.21.15.50:0/2503943827 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dddb1640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d815b3c0 0x7ff0d8147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dcdaf640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d80ca640 0x7ff0d815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 -- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d80ca640 msgr2=0x7ff0d815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d80ca640 0x7ff0d815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 -- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d815b3c0 msgr2=0x7ff0d8147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d815b3c0 0x7ff0d8147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 -- 172.21.15.50:0/2503943827 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d80c90a0 con 0x7ff0c4004d40 2024-04-02T16:22:12.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dddb1640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d815b3c0 0x7ff0d8147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.371+0000 7ff0dd5b0640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0d80ca100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40119d0 tx=0x7ff0d4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.372+0000 7ff0cdffb640 1 -- 172.21.15.50:0/2503943827 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400fb50 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.372+0000 7ff0cdffb640 1 -- 172.21.15.50:0/2503943827 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d400fcf0 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.372+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d80c9340 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.372+0000 7ff0cdffb640 1 -- 172.21.15.50:0/2503943827 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400fea0 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.372+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d80c9900 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.373+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d815b7d0 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.373+0000 7ff0cdffb640 1 -- 172.21.15.50:0/2503943827 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0d4012040 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.374+0000 7ff0cdffb640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff0bc06c920 0x7ff0bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.374+0000 7ff0cdffb640 1 -- 172.21.15.50:0/2503943827 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff0d408c5e0 con 0x7ff0c4004d40 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.374+0000 7ff0dcdaf640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff0bc06c920 0x7ff0bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.374+0000 7ff0dcdaf640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff0bc06c920 0x7ff0bc06ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff0d814dc50 tx=0x7ff0c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.378+0000 7ff0cdffb640 1 -- 172.21.15.50:0/2503943827 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff0d815b7d0 con 0x7ff0c4004d40 2024-04-02T16:22:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.563+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff0d806c640 con 0x7ff0c4004d40 2024-04-02T16:22:12.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.563+0000 7ff0cdffb640 1 -- 172.21.15.50:0/2503943827 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7ff0d4019030 con 0x7ff0c4004d40 2024-04-02T16:22:12.564 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:12.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff0bc06c920 msgr2=0x7ff0bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff0bc06c920 0x7ff0bc06ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff0d814dc50 tx=0x7ff0c8009040 comp rx=0 tx=0).stop 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 msgr2=0x7ff0d80ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0d80ca100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40119d0 tx=0x7ff0d4011e90 comp rx=0 tx=0).stop 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 shutdown_connections 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d815b3c0 0x7ff0d8147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d80ca640 0x7ff0d815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff0bc06c920 0x7ff0bc06ede0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 --2- 172.21.15.50:0/2503943827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c4004d40 0x7ff0d80ca100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 >> 172.21.15.50:0/2503943827 conn(0x7ff0d809d5e0 msgr2=0x7ff0d80b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 shutdown_connections 2024-04-02T16:22:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.565+0000 7ff0df814640 1 -- 172.21.15.50:0/2503943827 wait complete. 2024-04-02T16:22:12.577 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:22:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.678+0000 7f2a96e22640 1 Processor -- start 2024-04-02T16:22:12.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a96e22640 1 -- start start 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a96e22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9015c340 0x7f2a900b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a96e22640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2a9006f630 0x7f2a900cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a96e22640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9006fab0 con 0x7f2a9015c720 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a96e22640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f2a900d0710 con 0x7f2a9006f630 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a96e22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a900cd030 con 0x7f2a9015c340 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a953bf640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2a9006f630 0x7f2a900cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a8ffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2a9015c720 0x7f2a900cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49520/0 (socket says 172.21.15.50:49520) 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a953bf640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2a9006f630 0x7f2a900cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60334/0 (socket says 172.21.15.50:60334) 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.679+0000 7f2a8ffff640 1 -- 172.21.15.50:0/787786834 learned_addr learned my addr 172.21.15.50:0/787786834 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.680+0000 7f2a94bbe640 1 --2- 172.21.15.50:0/787786834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9015c340 0x7f2a900b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.680+0000 7f2a953bf640 1 -- 172.21.15.50:0/787786834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9015c340 msgr2=0x7f2a900b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:12.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.680+0000 7f2a953bf640 1 --2- 172.21.15.50:0/787786834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9015c340 0x7f2a900b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.680+0000 7f2a953bf640 1 -- 172.21.15.50:0/787786834 >> v1:172.21.15.50:6789/0 conn(0x7f2a9015c720 legacy=0x7f2a900cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.680+0000 7f2a8f7fe640 1 -- 172.21.15.50:0/787786834 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3976747789 0 0) 0x7f2a7c001110 con 0x7f2a9015c720 2024-04-02T16:22:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.680+0000 7f2a953bf640 1 -- 172.21.15.50:0/787786834 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a900ce110 con 0x7f2a9006f630 2024-04-02T16:22:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a953bf640 1 --2- 172.21.15.50:0/787786834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9006f630 0x7f2a900cdbd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009d90 tx=0x7f2a8802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d525f8280af99eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a8f7fe640 1 -- 172.21.15.50:0/787786834 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88039070 con 0x7f2a9006f630 2024-04-02T16:22:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a8f7fe640 1 -- 172.21.15.50:0/787786834 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a8802cc50 con 0x7f2a9006f630 2024-04-02T16:22:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a8f7fe640 1 -- 172.21.15.50:0/787786834 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88034440 con 0x7f2a9006f630 2024-04-02T16:22:12.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 -- 172.21.15.50:0/787786834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9006f630 msgr2=0x7f2a900cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 --2- 172.21.15.50:0/787786834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9006f630 0x7f2a900cdbd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009d90 tx=0x7f2a8802bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 -- 172.21.15.50:0/787786834 shutdown_connections 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 --2- 172.21.15.50:0/787786834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9006f630 0x7f2a900cdbd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 --2- 172.21.15.50:0/787786834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9015c340 0x7f2a900b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 -- 172.21.15.50:0/787786834 >> 172.21.15.50:0/787786834 conn(0x7f2a900b3c20 msgr2=0x7f2a900b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 -- 172.21.15.50:0/787786834 shutdown_connections 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.681+0000 7f2a96e22640 1 -- 172.21.15.50:0/787786834 wait complete. 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.682+0000 7f2a96e22640 1 Processor -- start 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a96e22640 1 -- start start 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a96e22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9006f630 0x7f2a900c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a96e22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9015c340 0x7f2a900c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a96e22640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9015c720 0x7f2a900c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a96e22640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2a900d0f20 con 0x7f2a9015c720 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a96e22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a900d0c40 con 0x7f2a9015c340 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a96e22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a900ccc20 con 0x7f2a9006f630 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.683+0000 7f2a953bf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9015c720 0x7f2a900c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9015c720 0x7f2a900c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60336/0 (socket says 172.21.15.50:60336) 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 -- 172.21.15.50:0/2033883155 learned_addr learned my addr 172.21.15.50:0/2033883155 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a94bbe640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9006f630 0x7f2a900c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a8ffff640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9015c340 0x7f2a900c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 -- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9006f630 msgr2=0x7f2a900c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9006f630 0x7f2a900c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 -- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9015c340 msgr2=0x7f2a900c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9015c340 0x7f2a900c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 -- 172.21.15.50:0/2033883155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a900c7490 con 0x7f2a9015c720 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a953bf640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9015c720 0x7f2a900c7e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009d90 tx=0x7f2a88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a8d7fa640 1 -- 172.21.15.50:0/2033883155 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88039050 con 0x7f2a9015c720 2024-04-02T16:22:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a8d7fa640 1 -- 172.21.15.50:0/2033883155 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a88045070 con 0x7f2a9015c720 2024-04-02T16:22:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a900c5eb0 con 0x7f2a9015c720 2024-04-02T16:22:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a8d7fa640 1 -- 172.21.15.50:0/2033883155 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8803e610 con 0x7f2a9015c720 2024-04-02T16:22:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.684+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a900c6470 con 0x7f2a9015c720 2024-04-02T16:22:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.685+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a90075760 con 0x7f2a9015c720 2024-04-02T16:22:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.686+0000 7f2a8d7fa640 1 -- 172.21.15.50:0/2033883155 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a8803e7b0 con 0x7f2a9015c720 2024-04-02T16:22:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.686+0000 7f2a8d7fa640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2a7406c920 0x7f2a7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.686+0000 7f2a8d7fa640 1 -- 172.21.15.50:0/2033883155 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2a880aeab0 con 0x7f2a9015c720 2024-04-02T16:22:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.686+0000 7f2a94bbe640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2a7406c920 0x7f2a7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.687+0000 7f2a94bbe640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2a7406c920 0x7f2a7406ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006fd0 tx=0x7f2a84008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.690+0000 7f2a8d7fa640 1 -- 172.21.15.50:0/2033883155 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2a900ce110 con 0x7f2a9015c720 2024-04-02T16:22:12.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.887+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2a900b4780 con 0x7f2a9015c720 2024-04-02T16:22:12.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.887+0000 7f2a8d7fa640 1 -- 172.21.15.50:0/2033883155 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f2a8807cc30 con 0x7f2a9015c720 2024-04-02T16:22:12.889 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2a7406c920 msgr2=0x7f2a7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2a7406c920 0x7f2a7406ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006fd0 tx=0x7f2a84008040 comp rx=0 tx=0).stop 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9015c720 msgr2=0x7f2a900c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9015c720 0x7f2a900c7e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009d90 tx=0x7f2a88004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 shutdown_connections 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2a7406c920 0x7f2a7406ede0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a9015c720 0x7f2a900c7e20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9015c340 0x7f2a900c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 --2- 172.21.15.50:0/2033883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9006f630 0x7f2a900c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 >> 172.21.15.50:0/2033883155 conn(0x7f2a900b3c20 msgr2=0x7f2a900b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 shutdown_connections 2024-04-02T16:22:12.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:12.889+0000 7f2a96e22640 1 -- 172.21.15.50:0/2033883155 wait complete. 2024-04-02T16:22:12.901 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:12.901 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:12.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:13.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.056+0000 7fd5b2b75640 1 Processor -- start 2024-04-02T16:22:13.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.057+0000 7fd5b2b75640 1 -- start start 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.057+0000 7fd5b2b75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac0d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.057+0000 7fd5b2b75640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd5ac06d800 0x7fd5ac09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.057+0000 7fd5b2b75640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ac06c760 con 0x7fd5ac0d12f0 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.057+0000 7fd5b2b75640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ac06d370 con 0x7fd5ac06d800 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.057+0000 7fd5b2b75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac06ca40 con 0x7fd5ac15e370 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.057+0000 7fd5a3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd5ac06d800 0x7fd5ac09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5a3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd5ac06d800 0x7fd5ac09aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60352/0 (socket says 172.21.15.50:60352) 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5a3fff640 1 -- 172.21.15.50:0/3840635395 learned_addr learned my addr 172.21.15.50:0/3840635395 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5b0911640 1 --2- 172.21.15.50:0/3840635395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac0d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5a3fff640 1 -- 172.21.15.50:0/3840635395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 msgr2=0x7fd5ac0d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5a3fff640 1 --2- 172.21.15.50:0/3840635395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac0d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5a3fff640 1 -- 172.21.15.50:0/3840635395 >> v1:172.21.15.188:6789/0 conn(0x7fd5ac0d12f0 legacy=0x7fd5ac162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:13.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5a3fff640 1 -- 172.21.15.50:0/3840635395 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac06cd20 con 0x7fd5ac06d800 2024-04-02T16:22:13.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.058+0000 7fd5a3fff640 1 --2- 172.21.15.50:0/3840635395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac09aa40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009870 tx=0x7fd59c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0faf5897455658 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5a37fe640 1 -- 172.21.15.50:0/3840635395 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c035020 con 0x7fd5ac06d800 2024-04-02T16:22:13.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5a37fe640 1 -- 172.21.15.50:0/3840635395 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59c031640 con 0x7fd5ac06d800 2024-04-02T16:22:13.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5a37fe640 1 -- 172.21.15.50:0/3840635395 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c02d040 con 0x7fd5ac06d800 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 -- 172.21.15.50:0/3840635395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 msgr2=0x7fd5ac09aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/3840635395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac09aa40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009870 tx=0x7fd59c02b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 -- 172.21.15.50:0/3840635395 shutdown_connections 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/3840635395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac09aa40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/3840635395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac0d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 -- 172.21.15.50:0/3840635395 >> 172.21.15.50:0/3840635395 conn(0x7fd5ac0b3ba0 msgr2=0x7fd5ac0b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 -- 172.21.15.50:0/3840635395 shutdown_connections 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.059+0000 7fd5b2b75640 1 -- 172.21.15.50:0/3840635395 wait complete. 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.060+0000 7fd5b2b75640 1 Processor -- start 2024-04-02T16:22:13.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.060+0000 7fd5b2b75640 1 -- start start 2024-04-02T16:22:13.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b2b75640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b2b75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5ac0d12f0 0x7fd5ac1567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b2b75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b2b75640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac15e8d0 con 0x7fd5ac06d800 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b2b75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac15e750 con 0x7fd5ac0d12f0 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b2b75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac09af80 con 0x7fd5ac15e370 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b0911640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b0911640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac1f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60358/0 (socket says 172.21.15.50:60358) 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.061+0000 7fd5b0911640 1 -- 172.21.15.50:0/2207143176 learned_addr learned my addr 172.21.15.50:0/2207143176 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5a3fff640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5ac0d12f0 0x7fd5ac1567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b1112640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b0911640 1 -- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 msgr2=0x7fd5ac15c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b0911640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac15c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b0911640 1 -- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5ac0d12f0 msgr2=0x7fd5ac1567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b0911640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5ac0d12f0 0x7fd5ac1567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b0911640 1 -- 172.21.15.50:0/2207143176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac0b5dc0 con 0x7fd5ac06d800 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5a3fff640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5ac0d12f0 0x7fd5ac1567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b0911640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac1f4190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800d5f0 tx=0x7fd5a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5a17fa640 1 -- 172.21.15.50:0/2207143176 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80125f0 con 0x7fd5ac06d800 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5a17fa640 1 -- 172.21.15.50:0/2207143176 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a8019070 con 0x7fd5ac06d800 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac0b60a0 con 0x7fd5ac06d800 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5a17fa640 1 -- 172.21.15.50:0/2207143176 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80129d0 con 0x7fd5ac06d800 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.062+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac0b6660 con 0x7fd5ac06d800 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.063+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac0756f0 con 0x7fd5ac06d800 2024-04-02T16:22:13.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.064+0000 7fd5a17fa640 1 -- 172.21.15.50:0/2207143176 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5a8012b70 con 0x7fd5ac06d800 2024-04-02T16:22:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.064+0000 7fd5a17fa640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd59806c8f0 0x7fd59806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.064+0000 7fd5a17fa640 1 -- 172.21.15.50:0/2207143176 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd5a808c280 con 0x7fd5ac06d800 2024-04-02T16:22:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.064+0000 7fd5a3fff640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd59806c8f0 0x7fd59806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.065+0000 7fd5a3fff640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd59806c8f0 0x7fd59806edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd59c036000 tx=0x7fd59c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.068+0000 7fd5a17fa640 1 -- 172.21.15.50:0/2207143176 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd5a805a400 con 0x7fd5ac06d800 2024-04-02T16:22:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.257+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5ac0b8030 con 0x7fd5ac06d800 2024-04-02T16:22:13.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.258+0000 7fd5a17fa640 1 -- 172.21.15.50:0/2207143176 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fd5a8059d00 con 0x7fd5ac06d800 2024-04-02T16:22:13.260 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:13.260 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd59806c8f0 msgr2=0x7fd59806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd59806c8f0 0x7fd59806edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd59c036000 tx=0x7fd59c03c040 comp rx=0 tx=0).stop 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 msgr2=0x7fd5ac1f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac1f4190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800d5f0 tx=0x7fd5a800dab0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 shutdown_connections 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5ac15e370 0x7fd5ac15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5ac0d12f0 0x7fd5ac1567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd59806c8f0 0x7fd59806edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 --2- 172.21.15.50:0/2207143176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5ac06d800 0x7fd5ac1f4190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 >> 172.21.15.50:0/2207143176 conn(0x7fd5ac0b3ba0 msgr2=0x7fd5ac157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 shutdown_connections 2024-04-02T16:22:13.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.260+0000 7fd5b2b75640 1 -- 172.21.15.50:0/2207143176 wait complete. 2024-04-02T16:22:13.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:13.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:13.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.377+0000 7f0d8e160640 1 Processor -- start 2024-04-02T16:22:13.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.377+0000 7f0d8e160640 1 -- start start 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.377+0000 7f0d8e160640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d880d4980 0x7f0d88162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.377+0000 7f0d8e160640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0d880d0c80 0x7f0d880d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.377+0000 7f0d8e160640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d880d15d0 con 0x7f0d88161ca0 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.377+0000 7f0d8e160640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f0d881630f0 con 0x7f0d880d0c80 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.377+0000 7f0d8e160640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0d880d5000 con 0x7f0d880d4980 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d87fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0d880d0c80 0x7f0d880d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d87fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0d880d0c80 0x7f0d880d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60364/0 (socket says 172.21.15.50:60364) 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d87fff640 1 -- 172.21.15.50:0/894484418 learned_addr learned my addr 172.21.15.50:0/894484418 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d86ffd640 1 --2- 172.21.15.50:0/894484418 >> v2:172.21.15.188:3300/0 conn(0x7f0d880d4980 0x7f0d88162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d87fff640 1 -- 172.21.15.50:0/894484418 >> v2:172.21.15.188:3300/0 conn(0x7f0d880d4980 msgr2=0x7f0d88162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:13.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d87fff640 1 --2- 172.21.15.50:0/894484418 >> v2:172.21.15.188:3300/0 conn(0x7f0d880d4980 0x7f0d88162bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d87fff640 1 -- 172.21.15.50:0/894484418 >> v1:172.21.15.188:6789/0 conn(0x7f0d88161ca0 legacy=0x7f0d880cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d87fff640 1 -- 172.21.15.50:0/894484418 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d880cf0d0 con 0x7f0d880d0c80 2024-04-02T16:22:13.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.378+0000 7f0d86ffd640 1 --2- 172.21.15.50:0/894484418 >> v2:172.21.15.188:3300/0 conn(0x7f0d880d4980 0x7f0d88162bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:13.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d87fff640 1 --2- 172.21.15.50:0/894484418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d0c80 0x7f0d880d1090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d8806bd50 tx=0x7f0d8002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1211da7a5fd06f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d867fc640 1 -- 172.21.15.50:0/894484418 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8002c9f0 con 0x7f0d880d0c80 2024-04-02T16:22:13.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d867fc640 1 -- 172.21.15.50:0/894484418 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8002cb90 con 0x7f0d880d0c80 2024-04-02T16:22:13.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d867fc640 1 -- 172.21.15.50:0/894484418 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d800343f0 con 0x7f0d880d0c80 2024-04-02T16:22:13.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 -- 172.21.15.50:0/894484418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d0c80 msgr2=0x7f0d880d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 --2- 172.21.15.50:0/894484418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d0c80 0x7f0d880d1090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d8806bd50 tx=0x7f0d8002be50 comp rx=0 tx=0).stop 2024-04-02T16:22:13.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 -- 172.21.15.50:0/894484418 shutdown_connections 2024-04-02T16:22:13.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 --2- 172.21.15.50:0/894484418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d0c80 0x7f0d880d1090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 --2- 172.21.15.50:0/894484418 >> v2:172.21.15.188:3300/0 conn(0x7f0d880d4980 0x7f0d88162bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 -- 172.21.15.50:0/894484418 >> 172.21.15.50:0/894484418 conn(0x7f0d8809d5c0 msgr2=0x7f0d880d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:13.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 -- 172.21.15.50:0/894484418 shutdown_connections 2024-04-02T16:22:13.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.379+0000 7f0d8e160640 1 -- 172.21.15.50:0/894484418 wait complete. 2024-04-02T16:22:13.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.381+0000 7f0d8e160640 1 Processor -- start 2024-04-02T16:22:13.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.381+0000 7f0d8e160640 1 -- start start 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.381+0000 7f0d8e160640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d880d0c80 0x7f0d88149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.381+0000 7f0d8e160640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d4980 0x7f0d880cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.381+0000 7f0d8e160640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d88161ca0 0x7f0d880cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.381+0000 7f0d8e160640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880ced80 con 0x7f0d880d4980 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d8e160640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880cf690 con 0x7f0d880d0c80 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d8e160640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0d880b70c0 con 0x7f0d88161ca0 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d4980 0x7f0d880cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d4980 0x7f0d880cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60380/0 (socket says 172.21.15.50:60380) 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 -- 172.21.15.50:0/2058190500 learned_addr learned my addr 172.21.15.50:0/2058190500 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d877fe640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d880d0c80 0x7f0d88149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d87fff640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d88161ca0 0x7f0d880cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 -- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d88161ca0 msgr2=0x7f0d880cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d88161ca0 0x7f0d880cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 -- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d880d0c80 msgr2=0x7f0d88149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d880d0c80 0x7f0d88149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 -- 172.21.15.50:0/2058190500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d880cbb90 con 0x7f0d880d4980 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d86ffd640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d4980 0x7f0d880cdaf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00e7e0 tx=0x7f0d7c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d6ffff640 1 -- 172.21.15.50:0/2058190500 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00f040 con 0x7f0d880d4980 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d6ffff640 1 -- 172.21.15.50:0/2058190500 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c004030 con 0x7f0d880d4980 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d880ce1f0 con 0x7f0d880d4980 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d6ffff640 1 -- 172.21.15.50:0/2058190500 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c004400 con 0x7f0d880d4980 2024-04-02T16:22:13.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.382+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881e8fd0 con 0x7f0d880d4980 2024-04-02T16:22:13.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.383+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d88075760 con 0x7f0d880d4980 2024-04-02T16:22:13.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.384+0000 7f0d6ffff640 1 -- 172.21.15.50:0/2058190500 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d7c00c6d0 con 0x7f0d880d4980 2024-04-02T16:22:13.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.384+0000 7f0d6ffff640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0d7406c8f0 0x7f0d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.384+0000 7f0d6ffff640 1 -- 172.21.15.50:0/2058190500 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0d7c08bb70 con 0x7f0d880d4980 2024-04-02T16:22:13.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.385+0000 7f0d877fe640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0d7406c8f0 0x7f0d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.385+0000 7f0d877fe640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0d7406c8f0 0x7f0d7406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000f60 tx=0x7f0d78009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.389+0000 7f0d6ffff640 1 -- 172.21.15.50:0/2058190500 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0d7c017020 con 0x7f0d880d4980 2024-04-02T16:22:13.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.574+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d880ba130 con 0x7f0d880d4980 2024-04-02T16:22:13.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.574+0000 7f0d6ffff640 1 -- 172.21.15.50:0/2058190500 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f0d7c059610 con 0x7f0d880d4980 2024-04-02T16:22:13.576 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0d7406c8f0 msgr2=0x7f0d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0d7406c8f0 0x7f0d7406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000f60 tx=0x7f0d78009040 comp rx=0 tx=0).stop 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d4980 msgr2=0x7f0d880cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d4980 0x7f0d880cdaf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00e7e0 tx=0x7f0d7c00eca0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 shutdown_connections 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d88161ca0 0x7f0d880cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0d7406c8f0 0x7f0d7406edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d880d4980 0x7f0d880cdaf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 --2- 172.21.15.50:0/2058190500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d880d0c80 0x7f0d88149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 >> 172.21.15.50:0/2058190500 conn(0x7f0d8809d5c0 msgr2=0x7f0d8806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 shutdown_connections 2024-04-02T16:22:13.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.576+0000 7f0d8e160640 1 -- 172.21.15.50:0/2058190500 wait complete. 2024-04-02T16:22:13.588 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:13.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.692+0000 7f973fa24640 1 Processor -- start 2024-04-02T16:22:13.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.692+0000 7f973fa24640 1 -- start start 2024-04-02T16:22:13.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973fa24640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738141500 0x7f9738140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973fa24640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9738140e60 0x7f9738071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973fa24640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97380cf450 con 0x7f973813e120 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973fa24640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f97381412b0 con 0x7f9738140e60 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973fa24640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97380cf730 con 0x7f9738141500 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973dfc1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9738140e60 0x7f9738071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973d7c0640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f973813e120 0x7f97380cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:49530/0 (socket says 172.21.15.50:49530) 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.693+0000 7f973d7c0640 1 -- 172.21.15.50:0/1408940409 learned_addr learned my addr 172.21.15.50:0/1408940409 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f973cfbf640 1 --2- 172.21.15.50:0/1408940409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738141500 0x7f9738140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f973dfc1640 1 -- 172.21.15.50:0/1408940409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738141500 msgr2=0x7f9738140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f973dfc1640 1 --2- 172.21.15.50:0/1408940409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738141500 0x7f9738140920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f973dfc1640 1 -- 172.21.15.50:0/1408940409 >> v1:172.21.15.50:6789/0 conn(0x7f973813e120 legacy=0x7f97380cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f973dfc1640 1 -- 172.21.15.50:0/1408940409 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97380cfa10 con 0x7f9738140e60 2024-04-02T16:22:13.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f9737fff640 1 -- 172.21.15.50:0/1408940409 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617758688 0 0) 0x7f97380cf450 con 0x7f973813e120 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f973dfc1640 1 --2- 172.21.15.50:0/1408940409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738140e60 0x7f9738071920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9728009d90 tx=0x7f972802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65640e39826989b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f9737fff640 1 -- 172.21.15.50:0/1408940409 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728039070 con 0x7f9738140e60 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f9737fff640 1 -- 172.21.15.50:0/1408940409 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972802cc50 con 0x7f9738140e60 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.694+0000 7f9737fff640 1 -- 172.21.15.50:0/1408940409 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728034440 con 0x7f9738140e60 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 -- 172.21.15.50:0/1408940409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738140e60 msgr2=0x7f9738071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 --2- 172.21.15.50:0/1408940409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738140e60 0x7f9738071920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9728009d90 tx=0x7f972802bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 -- 172.21.15.50:0/1408940409 shutdown_connections 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 --2- 172.21.15.50:0/1408940409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738140e60 0x7f9738071920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 --2- 172.21.15.50:0/1408940409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738141500 0x7f9738140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 -- 172.21.15.50:0/1408940409 >> 172.21.15.50:0/1408940409 conn(0x7f973806eb90 msgr2=0x7f973806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 -- 172.21.15.50:0/1408940409 shutdown_connections 2024-04-02T16:22:13.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.695+0000 7f973fa24640 1 -- 172.21.15.50:0/1408940409 wait complete. 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.696+0000 7f973fa24640 1 Processor -- start 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.696+0000 7f973fa24640 1 -- start start 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973fa24640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f973813e120 0x7f97380aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973fa24640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738140e60 0x7f97381546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973fa24640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738141500 0x7f9738154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973fa24640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f973813e500 con 0x7f9738141500 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973fa24640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9738071400 con 0x7f973813e120 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973fa24640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97380cf110 con 0x7f9738140e60 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738141500 0x7f9738154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738141500 0x7f9738154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60406/0 (socket says 172.21.15.50:60406) 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 -- 172.21.15.50:0/873617046 learned_addr learned my addr 172.21.15.50:0/873617046 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973cfbf640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738140e60 0x7f97381546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973d7c0640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f973813e120 0x7f97380aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 -- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738140e60 msgr2=0x7f97381546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738140e60 0x7f97381546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 -- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f973813e120 msgr2=0x7f97380aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f973813e120 0x7f97380aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 -- 172.21.15.50:0/873617046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9738152df0 con 0x7f9738141500 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973cfbf640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738140e60 0x7f97381546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.697+0000 7f973dfc1640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738141500 0x7f9738154c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9728037040 tx=0x7f9728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.698+0000 7f9735ffb640 1 -- 172.21.15.50:0/873617046 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728039050 con 0x7f9738141500 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.698+0000 7f9735ffb640 1 -- 172.21.15.50:0/873617046 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972803a040 con 0x7f9738141500 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.698+0000 7f9735ffb640 1 -- 172.21.15.50:0/873617046 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972802ca50 con 0x7f9738141500 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.698+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97381530d0 con 0x7f9738141500 2024-04-02T16:22:13.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.698+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9738153690 con 0x7f9738141500 2024-04-02T16:22:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.699+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9738075760 con 0x7f9738141500 2024-04-02T16:22:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.699+0000 7f9735ffb640 1 -- 172.21.15.50:0/873617046 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9728047030 con 0x7f9738141500 2024-04-02T16:22:13.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.700+0000 7f9735ffb640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f972406c8f0 0x7f972406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:13.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.700+0000 7f9735ffb640 1 -- 172.21.15.50:0/873617046 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f97280ae030 con 0x7f9738141500 2024-04-02T16:22:13.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.700+0000 7f973d7c0640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f972406c8f0 0x7f972406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:13.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.701+0000 7f973d7c0640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f972406c8f0 0x7f972406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9730000f60 tx=0x7f9730007b70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:13.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.704+0000 7f9735ffb640 1 -- 172.21.15.50:0/873617046 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f972804e060 con 0x7f9738141500 2024-04-02T16:22:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.890+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97380734f0 con 0x7f9738141500 2024-04-02T16:22:13.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.890+0000 7f9735ffb640 1 -- 172.21.15.50:0/873617046 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f972807c190 con 0x7f9738141500 2024-04-02T16:22:13.891 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:13.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f972406c8f0 msgr2=0x7f972406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f972406c8f0 0x7f972406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9730000f60 tx=0x7f9730007b70 comp rx=0 tx=0).stop 2024-04-02T16:22:13.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738141500 msgr2=0x7f9738154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738141500 0x7f9738154c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9728037040 tx=0x7f9728004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 shutdown_connections 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f972406c8f0 0x7f972406edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9738141500 0x7f9738154c10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9738140e60 0x7f97381546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 --2- 172.21.15.50:0/873617046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f973813e120 0x7f97380aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 >> 172.21.15.50:0/873617046 conn(0x7f973806eb90 msgr2=0x7f973813ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 shutdown_connections 2024-04-02T16:22:13.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:13.892+0000 7f973fa24640 1 -- 172.21.15.50:0/873617046 wait complete. 2024-04-02T16:22:13.904 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:14.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.008+0000 7f7c52d9d640 1 Processor -- start 2024-04-02T16:22:14.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.008+0000 7f7c52d9d640 1 -- start start 2024-04-02T16:22:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c52d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7c4c0d0ab0 0x7f7c4c1628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c15fe10 0x7f7c4c160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c52d9d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c4c0b5970 con 0x7f7c4c0d49c0 2024-04-02T16:22:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c52d9d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4c0cede0 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c52d9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4c0cf0c0 con 0x7f7c4c15fe10 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c51d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7c4c0d0ab0 0x7f7c4c1628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c51d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7c4c0d0ab0 0x7f7c4c1628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60410/0 (socket says 172.21.15.50:60410) 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c51d9b640 1 -- 172.21.15.50:0/3675942948 learned_addr learned my addr 172.21.15.50:0/3675942948 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c5259c640 1 --2- 172.21.15.50:0/3675942948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c15fe10 0x7f7c4c160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c51d9b640 1 -- 172.21.15.50:0/3675942948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c15fe10 msgr2=0x7f7c4c160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c51d9b640 1 --2- 172.21.15.50:0/3675942948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c15fe10 0x7f7c4c160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c51d9b640 1 -- 172.21.15.50:0/3675942948 >> v1:172.21.15.188:6789/0 conn(0x7f7c4c0d49c0 legacy=0x7f7c4c0d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c51d9b640 1 -- 172.21.15.50:0/3675942948 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c0cf3a0 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.009+0000 7f7c5259c640 1 --2- 172.21.15.50:0/3675942948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c15fe10 0x7f7c4c160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.010+0000 7f7c51d9b640 1 --2- 172.21.15.50:0/3675942948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1628b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c4402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8971329d27a150e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.010+0000 7f7c50d99640 1 -- 172.21.15.50:0/3675942948 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44034070 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.010+0000 7f7c50d99640 1 -- 172.21.15.50:0/3675942948 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c44030840 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.010+0000 7f7c50d99640 1 -- 172.21.15.50:0/3675942948 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44030c10 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 -- 172.21.15.50:0/3675942948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 msgr2=0x7f7c4c1628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/3675942948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1628b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c4402b980 comp rx=0 tx=0).stop 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 -- 172.21.15.50:0/3675942948 shutdown_connections 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/3675942948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c15fe10 0x7f7c4c160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/3675942948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1628b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 -- 172.21.15.50:0/3675942948 >> 172.21.15.50:0/3675942948 conn(0x7f7c4c09d410 msgr2=0x7f7c4c0d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 -- 172.21.15.50:0/3675942948 shutdown_connections 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.011+0000 7f7c52d9d640 1 -- 172.21.15.50:0/3675942948 wait complete. 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.012+0000 7f7c52d9d640 1 Processor -- start 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.012+0000 7f7c52d9d640 1 -- start start 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c0d49c0 0x7f7c4c0a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c15fe10 0x7f7c4c0a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c52d9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0cfe80 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c52d9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0cf960 con 0x7f7c4c15fe10 2024-04-02T16:22:14.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c52d9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4c1f5750 con 0x7f7c4c0d49c0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60414/0 (socket says 172.21.15.50:60414) 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 -- 172.21.15.50:0/4074846717 learned_addr learned my addr 172.21.15.50:0/4074846717 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c5159a640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c0d49c0 0x7f7c4c0a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c5259c640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c15fe10 0x7f7c4c0a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 -- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c0d49c0 msgr2=0x7f7c4c0a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c0d49c0 0x7f7c4c0a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 -- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c15fe10 msgr2=0x7f7c4c0a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c15fe10 0x7f7c4c0a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 -- 172.21.15.50:0/4074846717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c1f8160 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c51d9b640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1f5210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.013+0000 7f7c4a7fc640 1 -- 172.21.15.50:0/4074846717 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44048070 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.014+0000 7f7c4a7fc640 1 -- 172.21.15.50:0/4074846717 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c44030640 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.014+0000 7f7c4a7fc640 1 -- 172.21.15.50:0/4074846717 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44034040 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.014+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c1f8440 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.014+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1f8a00 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.014+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4c0a1ff0 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.015+0000 7f7c4a7fc640 1 -- 172.21.15.50:0/4074846717 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c4404b070 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.015+0000 7f7c4a7fc640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c3806c8f0 0x7f7c3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.015+0000 7f7c4a7fc640 1 -- 172.21.15.50:0/4074846717 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7c440adf20 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.016+0000 7f7c5159a640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c3806c8f0 0x7f7c3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.016+0000 7f7c5159a640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c3806c8f0 0x7f7c3806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0cb0d0 tx=0x7f7c40009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.020+0000 7f7c4a7fc640 1 -- 172.21.15.50:0/4074846717 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7c4407c0a0 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.202+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c4c1f5910 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.202+0000 7f7c4a7fc640 1 -- 172.21.15.50:0/4074846717 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f7c4407b9a0 con 0x7f7c4c0d0ab0 2024-04-02T16:22:14.204 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c3806c8f0 msgr2=0x7f7c3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c3806c8f0 0x7f7c3806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0cb0d0 tx=0x7f7c40009040 comp rx=0 tx=0).stop 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 msgr2=0x7f7c4c1f5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1f5210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c44004730 comp rx=0 tx=0).stop 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 shutdown_connections 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c4c15fe10 0x7f7c4c0a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c4c0d49c0 0x7f7c4c0a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c3806c8f0 0x7f7c3806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 --2- 172.21.15.50:0/4074846717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c4c0d0ab0 0x7f7c4c1f5210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 >> 172.21.15.50:0/4074846717 conn(0x7f7c4c09d410 msgr2=0x7f7c4c15f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 shutdown_connections 2024-04-02T16:22:14.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.204+0000 7f7c52d9d640 1 -- 172.21.15.50:0/4074846717 wait complete. 2024-04-02T16:22:14.216 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:14.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.319+0000 7f881972b640 1 Processor -- start 2024-04-02T16:22:14.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.319+0000 7f881972b640 1 -- start start 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f881972b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8814071ea0 0x7f88140b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f881972b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8814072280 0x7f88140b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f881972b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f88140b5430 0x7f881415b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f881972b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f88140b4940 con 0x7f88140b5430 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f881972b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f881415c6d0 con 0x7f8814072280 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f881972b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f88140b3f70 con 0x7f8814071ea0 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f88140b5430 0x7f881415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f88140b5430 0x7f881415b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60418/0 (socket says 172.21.15.50:60418) 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 -- 172.21.15.50:0/3857519948 learned_addr learned my addr 172.21.15.50:0/3857519948 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:14.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f8812ffd640 1 --2- 172.21.15.50:0/3857519948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8814071ea0 0x7f88140b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88127fc640 1 --2- 172.21.15.50:0/3857519948 >> v2:172.21.15.188:3300/0 conn(0x7f8814072280 0x7f88140b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 -- 172.21.15.50:0/3857519948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8814071ea0 msgr2=0x7f88140b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 --2- 172.21.15.50:0/3857519948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8814071ea0 0x7f88140b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 -- 172.21.15.50:0/3857519948 >> v2:172.21.15.188:3300/0 conn(0x7f8814072280 msgr2=0x7f88140b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 --2- 172.21.15.50:0/3857519948 >> v2:172.21.15.188:3300/0 conn(0x7f8814072280 0x7f88140b4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.320+0000 7f88137fe640 1 -- 172.21.15.50:0/3857519948 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88140b58c0 con 0x7f88140b5430 2024-04-02T16:22:14.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.321+0000 7f88137fe640 1 --2- 172.21.15.50:0/3857519948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88140b5430 0x7f881415b690 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f880c009870 tx=0x7f880c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41b7fe9826b321d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.321+0000 7f8811ffb640 1 -- 172.21.15.50:0/3857519948 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0317c0 con 0x7f88140b5430 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.321+0000 7f8811ffb640 1 -- 172.21.15.50:0/3857519948 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c037070 con 0x7f88140b5430 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.321+0000 7f8811ffb640 1 -- 172.21.15.50:0/3857519948 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c031b80 con 0x7f88140b5430 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 -- 172.21.15.50:0/3857519948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88140b5430 msgr2=0x7f881415b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 --2- 172.21.15.50:0/3857519948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88140b5430 0x7f881415b690 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f880c009870 tx=0x7f880c02b860 comp rx=0 tx=0).stop 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 -- 172.21.15.50:0/3857519948 shutdown_connections 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 --2- 172.21.15.50:0/3857519948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88140b5430 0x7f881415b690 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 --2- 172.21.15.50:0/3857519948 >> v2:172.21.15.188:3300/0 conn(0x7f8814072280 0x7f88140b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 --2- 172.21.15.50:0/3857519948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8814071ea0 0x7f88140b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 -- 172.21.15.50:0/3857519948 >> 172.21.15.50:0/3857519948 conn(0x7f881406b520 msgr2=0x7f881406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 -- 172.21.15.50:0/3857519948 shutdown_connections 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.322+0000 7f881972b640 1 -- 172.21.15.50:0/3857519948 wait complete. 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.323+0000 7f881972b640 1 Processor -- start 2024-04-02T16:22:14.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.323+0000 7f881972b640 1 -- start start 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f881972b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8814071ea0 0x7f8814140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f881972b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8814072280 0x7f8814137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f881972b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88140b5430 0x7f881415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f881972b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88140b3c00 con 0x7f8814072280 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f881972b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88140b3920 con 0x7f8814071ea0 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f881972b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f881415bbd0 con 0x7f88140b5430 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8814072280 0x7f8814137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8814072280 0x7f8814137e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60430/0 (socket says 172.21.15.50:60430) 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 -- 172.21.15.50:0/3969039386 learned_addr learned my addr 172.21.15.50:0/3969039386 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88137fe640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88140b5430 0x7f881415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f8812ffd640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8814071ea0 0x7f8814140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 -- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88140b5430 msgr2=0x7f881415a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88140b5430 0x7f881415a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 -- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8814071ea0 msgr2=0x7f8814140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8814071ea0 0x7f8814140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f88127fc640 1 -- 172.21.15.50:0/3969039386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881413f2a0 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.324+0000 7f8812ffd640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8814071ea0 0x7f8814140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.325+0000 7f88127fc640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8814072280 0x7f8814137e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c7f0 tx=0x7f87fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.325+0000 7f880b7fe640 1 -- 172.21.15.50:0/3969039386 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc015070 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.325+0000 7f880b7fe640 1 -- 172.21.15.50:0/3969039386 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87fc004240 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.325+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881413f580 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.325+0000 7f880b7fe640 1 -- 172.21.15.50:0/3969039386 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc007620 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.325+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881413fb40 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.326+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8814140ea0 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.326+0000 7f880b7fe640 1 -- 172.21.15.50:0/3969039386 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87fc0079d0 con 0x7f8814072280 2024-04-02T16:22:14.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.327+0000 7f880b7fe640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f87f406c8f0 0x7f87f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.327+0000 7f880b7fe640 1 -- 172.21.15.50:0/3969039386 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f87fc08c160 con 0x7f8814072280 2024-04-02T16:22:14.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.327+0000 7f8812ffd640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f87f406c8f0 0x7f87f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.328+0000 7f8812ffd640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f87f406c8f0 0x7f87f406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f880400e4a0 tx=0x7f8804007650 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.331+0000 7f880b7fe640 1 -- 172.21.15.50:0/3969039386 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8814140ea0 con 0x7f8814072280 2024-04-02T16:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.513+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8814075760 con 0x7f8814072280 2024-04-02T16:22:14.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.514+0000 7f880b7fe640 1 -- 172.21.15.50:0/3969039386 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f87fc059c90 con 0x7f8814072280 2024-04-02T16:22:14.515 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f87f406c8f0 msgr2=0x7f87f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f87f406c8f0 0x7f87f406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f880400e4a0 tx=0x7f8804007650 comp rx=0 tx=0).stop 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8814072280 msgr2=0x7f8814137e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8814072280 0x7f8814137e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c7f0 tx=0x7f87fc00ccb0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 shutdown_connections 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88140b5430 0x7f881415a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f87f406c8f0 0x7f87f406edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8814072280 0x7f8814137e30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 --2- 172.21.15.50:0/3969039386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8814071ea0 0x7f8814140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 >> 172.21.15.50:0/3969039386 conn(0x7f881406b520 msgr2=0x7f88140b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 shutdown_connections 2024-04-02T16:22:14.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.516+0000 7f881972b640 1 -- 172.21.15.50:0/3969039386 wait complete. 2024-04-02T16:22:14.528 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:22:14.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.629+0000 7fcfefe0d640 1 Processor -- start 2024-04-02T16:22:14.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.630+0000 7fcfefe0d640 1 -- start start 2024-04-02T16:22:14.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.630+0000 7fcfefe0d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcfe8162600 0x7fcfe80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.630+0000 7fcfefe0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe81629e0 0x7fcfe80cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.630+0000 7fcfefe0d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe809b450 con 0x7fcfe8162f30 2024-04-02T16:22:14.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.630+0000 7fcfefe0d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe80d0aa0 con 0x7fcfe8162600 2024-04-02T16:22:14.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.630+0000 7fcfefe0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe809b6f0 con 0x7fcfe81629e0 2024-04-02T16:22:14.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfedba9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcfe8162600 0x7fcfe80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfedba9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcfe8162600 0x7fcfe80b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60438/0 (socket says 172.21.15.50:60438) 2024-04-02T16:22:14.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfedba9640 1 -- 172.21.15.50:0/2410514216 learned_addr learned my addr 172.21.15.50:0/2410514216 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:14.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfee3aa640 1 --2- 172.21.15.50:0/2410514216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe81629e0 0x7fcfe80cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfedba9640 1 -- 172.21.15.50:0/2410514216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe81629e0 msgr2=0x7fcfe80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:14.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfedba9640 1 --2- 172.21.15.50:0/2410514216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe81629e0 0x7fcfe80cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfedba9640 1 -- 172.21.15.50:0/2410514216 >> v1:172.21.15.188:6789/0 conn(0x7fcfe8162f30 legacy=0x7fcfe80d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.631+0000 7fcfedba9640 1 -- 172.21.15.50:0/2410514216 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe809b9d0 con 0x7fcfe8162600 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfedba9640 1 --2- 172.21.15.50:0/2410514216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80b4740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40098a0 tx=0x7fcfe402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c05ed264211af70 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfecba7640 1 -- 172.21.15.50:0/2410514216 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4034070 con 0x7fcfe8162600 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfecba7640 1 -- 172.21.15.50:0/2410514216 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4030840 con 0x7fcfe8162600 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfecba7640 1 -- 172.21.15.50:0/2410514216 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4030c10 con 0x7fcfe8162600 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 -- 172.21.15.50:0/2410514216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 msgr2=0x7fcfe80b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/2410514216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80b4740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40098a0 tx=0x7fcfe402b980 comp rx=0 tx=0).stop 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 -- 172.21.15.50:0/2410514216 shutdown_connections 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/2410514216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe81629e0 0x7fcfe80cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/2410514216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80b4740 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 -- 172.21.15.50:0/2410514216 >> 172.21.15.50:0/2410514216 conn(0x7fcfe80b3c20 msgr2=0x7fcfe80b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 -- 172.21.15.50:0/2410514216 shutdown_connections 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.632+0000 7fcfefe0d640 1 -- 172.21.15.50:0/2410514216 wait complete. 2024-04-02T16:22:14.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.633+0000 7fcfefe0d640 1 Processor -- start 2024-04-02T16:22:14.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfefe0d640 1 -- start start 2024-04-02T16:22:14.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfefe0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfefe0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe81629e0 0x7fcfe8152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfefe0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe8162f30 0x7fcfe8153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfefe0d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80d0350 con 0x7fcfe8162600 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfefe0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe80d1640 con 0x7fcfe81629e0 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfefe0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe80d04d0 con 0x7fcfe8162f30 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfedba9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfedba9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60444/0 (socket says 172.21.15.50:60444) 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfedba9640 1 -- 172.21.15.50:0/1126847757 learned_addr learned my addr 172.21.15.50:0/1126847757 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfee3aa640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe8162f30 0x7fcfe8153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.634+0000 7fcfed3a8640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe81629e0 0x7fcfe8152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfedba9640 1 -- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe8162f30 msgr2=0x7fcfe8153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfedba9640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe8162f30 0x7fcfe8153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfedba9640 1 -- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe81629e0 msgr2=0x7fcfe8152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfedba9640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe81629e0 0x7fcfe8152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfedba9640 1 -- 172.21.15.50:0/1126847757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe80b60f0 con 0x7fcfe8162600 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfed3a8640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe81629e0 0x7fcfe8152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfedba9640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80c9200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfe402b9c0 tx=0x7fcfe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfde7fc640 1 -- 172.21.15.50:0/1126847757 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4048070 con 0x7fcfe8162600 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfde7fc640 1 -- 172.21.15.50:0/1126847757 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4030640 con 0x7fcfe8162600 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe80b6390 con 0x7fcfe8162600 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfde7fc640 1 -- 172.21.15.50:0/1126847757 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4034040 con 0x7fcfe8162600 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.635+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe80b6950 con 0x7fcfe8162600 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.636+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe8075760 con 0x7fcfe8162600 2024-04-02T16:22:14.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.636+0000 7fcfde7fc640 1 -- 172.21.15.50:0/1126847757 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfe404b070 con 0x7fcfe8162600 2024-04-02T16:22:14.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.637+0000 7fcfde7fc640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcfcc06c8f0 0x7fcfcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:14.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.637+0000 7fcfde7fc640 1 -- 172.21.15.50:0/1126847757 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcfe40adcf0 con 0x7fcfe8162600 2024-04-02T16:22:14.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.637+0000 7fcfed3a8640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcfcc06c8f0 0x7fcfcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:14.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.638+0000 7fcfed3a8640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcfcc06c8f0 0x7fcfcc06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80cfa10 tx=0x7fcfd8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:14.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.641+0000 7fcfde7fc640 1 -- 172.21.15.50:0/1126847757 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcfe40b2070 con 0x7fcfe8162600 2024-04-02T16:22:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.838+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcfe80b4100 con 0x7fcfe8162600 2024-04-02T16:22:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.839+0000 7fcfde7fc640 1 -- 172.21.15.50:0/1126847757 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fcfe407b820 con 0x7fcfe8162600 2024-04-02T16:22:14.840 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-04-02T16:22:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcfcc06c8f0 msgr2=0x7fcfcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcfcc06c8f0 0x7fcfcc06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80cfa10 tx=0x7fcfd8009040 comp rx=0 tx=0).stop 2024-04-02T16:22:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 msgr2=0x7fcfe80c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:14.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80c9200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfe402b9c0 tx=0x7fcfe4004730 comp rx=0 tx=0).stop 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 shutdown_connections 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfe8162f30 0x7fcfe8153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe81629e0 0x7fcfe8152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcfcc06c8f0 0x7fcfcc06edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 --2- 172.21.15.50:0/1126847757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfe8162600 0x7fcfe80c9200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 >> 172.21.15.50:0/1126847757 conn(0x7fcfe80b3c20 msgr2=0x7fcfe815c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 shutdown_connections 2024-04-02T16:22:14.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:14.841+0000 7fcfefe0d640 1 -- 172.21.15.50:0/1126847757 wait complete. 2024-04-02T16:22:14.852 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2024-04-02T16:22:14.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:14.853 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-02T16:22:14.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[44150]: fuse finished with error 0 and tester_r 0 2024-04-02T16:22:14.880 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:22:20.982 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:22:20.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:20.982 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-02T16:22:21.005 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:21.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.157+0000 7fc27ed04640 1 Processor -- start 2024-04-02T16:22:21.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.157+0000 7fc27ed04640 1 -- start start 2024-04-02T16:22:21.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc27ed04640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc27815e670 0x7fc2780b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc27ed04640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780d0210 con 0x7fc27809aa40 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc27ed04640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc27806d480 con 0x7fc27806cff0 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc27ed04640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc27806da20 con 0x7fc27815e670 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc27d2a1640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc27809aa40 0x7fc27806f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47104/0 (socket says 172.21.15.50:47104) 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc27d2a1640 1 -- 172.21.15.50:0/1414419402 learned_addr learned my addr 172.21.15.50:0/1414419402 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc27caa0640 1 --2- 172.21.15.50:0/1414419402 >> v2:172.21.15.188:3300/0 conn(0x7fc27815e670 0x7fc2780b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3598745018 0 0) 0x7fc2780d0210 con 0x7fc27809aa40 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc260003520 con 0x7fc27809aa40 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1102035721 0 0) 0x7fc260003520 con 0x7fc27809aa40 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 >> v2:172.21.15.188:3300/0 conn(0x7fc27815e670 msgr2=0x7fc2780b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 --2- 172.21.15.50:0/1414419402 >> v2:172.21.15.188:3300/0 conn(0x7fc27815e670 0x7fc2780b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 >> v1:172.21.15.188:6789/0 conn(0x7fc27806cff0 legacy=0x7fc2780d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27806d740 con 0x7fc27809aa40 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fc2700019f0 con 0x7fc27809aa40 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 >> v1:172.21.15.50:6789/0 conn(0x7fc27809aa40 legacy=0x7fc27806f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2600039b0 0x7fc260003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc260004800 0x7fc260004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2600051c0 0x7fc26000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc27806da20 con 0x7fc260004800 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc26000c070 con 0x7fc2600051c0 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.158+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc26000c270 con 0x7fc2600039b0 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc26ffff640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc260004800 0x7fc260004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc27caa0640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2600039b0 0x7fc260003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc27d2a1640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2600051c0 0x7fc26000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc26ffff640 1 -- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2600039b0 msgr2=0x7fc260003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc26ffff640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2600039b0 0x7fc260003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc26ffff640 1 -- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2600051c0 msgr2=0x7fc26000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc26ffff640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2600051c0 0x7fc26000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc26ffff640 1 -- 172.21.15.50:0/1414419402 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc260003520 con 0x7fc260004800 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.159+0000 7fc27d2a1640 1 -- 172.21.15.50:0/1414419402 reap_dead start 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc26ffff640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc260004800 0x7fc260004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc268001750 tx=0x7fc260005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c11ec5ca93a114 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268001370 con 0x7fc260004800 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2680052d0 con 0x7fc260004800 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc26f7fe640 1 -- 172.21.15.50:0/1414419402 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268003830 con 0x7fc260004800 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc27ed04640 1 -- 172.21.15.50:0/1414419402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc260004800 msgr2=0x7fc260004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc27ed04640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc260004800 0x7fc260004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc268001750 tx=0x7fc260005830 comp rx=0 tx=0).stop 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc27ed04640 1 -- 172.21.15.50:0/1414419402 shutdown_connections 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc27ed04640 1 --2- 172.21.15.50:0/1414419402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc260004800 0x7fc260004c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc27ed04640 1 -- 172.21.15.50:0/1414419402 >> 172.21.15.50:0/1414419402 conn(0x7fc2780b3b90 msgr2=0x7fc2780b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc27ed04640 1 -- 172.21.15.50:0/1414419402 shutdown_connections 2024-04-02T16:22:21.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.160+0000 7fc27ed04640 1 -- 172.21.15.50:0/1414419402 wait complete. 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.161+0000 7fc27ed04640 1 Processor -- start 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.162+0000 7fc27ed04640 1 -- start start 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.162+0000 7fc27ed04640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc260004800 0x7fc27815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.162+0000 7fc27ed04640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc27814e470 0x7fc2780c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.162+0000 7fc27ed04640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27806f5a0 0x7fc27806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.162+0000 7fc27ed04640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780d0790 con 0x7fc27814e470 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.162+0000 7fc27ed04640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780d0b00 con 0x7fc260004800 2024-04-02T16:22:21.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.162+0000 7fc27ed04640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc2780b4680 con 0x7fc27806f5a0 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc27814e470 0x7fc2780c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc27814e470 0x7fc2780c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39962/0 (socket says 172.21.15.50:39962) 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 -- 172.21.15.50:0/3078384084 learned_addr learned my addr 172.21.15.50:0/3078384084 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc27caa0640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc260004800 0x7fc27815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc27d2a1640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27806f5a0 0x7fc27806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 -- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27806f5a0 msgr2=0x7fc27806fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27806f5a0 0x7fc27806fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 -- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc260004800 msgr2=0x7fc27815b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc260004800 0x7fc27815b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 -- 172.21.15.50:0/3078384084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2781f7b60 con 0x7fc27814e470 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc27caa0640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc260004800 0x7fc27815b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26ffff640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc27814e470 0x7fc2780c84f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc268000c00 tx=0x7fc268001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26d7fa640 1 -- 172.21.15.50:0/3078384084 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268041020 con 0x7fc27814e470 2024-04-02T16:22:21.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26d7fa640 1 -- 172.21.15.50:0/3078384084 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc268005480 con 0x7fc27814e470 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2781f7e40 con 0x7fc27814e470 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc26d7fa640 1 -- 172.21.15.50:0/3078384084 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268007a30 con 0x7fc27814e470 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.163+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2781f8400 con 0x7fc27814e470 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.164+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2780756f0 con 0x7fc27814e470 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.165+0000 7fc26d7fa640 1 -- 172.21.15.50:0/3078384084 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc268044020 con 0x7fc27814e470 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.165+0000 7fc26d7fa640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc26406c8f0 0x7fc26406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.165+0000 7fc26d7fa640 1 -- 172.21.15.50:0/3078384084 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc2680ad910 con 0x7fc27814e470 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.165+0000 7fc27caa0640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc26406c8f0 0x7fc26406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.166+0000 7fc27caa0640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc26406c8f0 0x7fc26406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc278069c20 tx=0x7fc274008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.170+0000 7fc26d7fa640 1 -- 172.21.15.50:0/3078384084 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc26804b030 con 0x7fc27814e470 2024-04-02T16:22:21.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.357+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2780b8020 con 0x7fc27814e470 2024-04-02T16:22:21.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.358+0000 7fc26d7fa640 1 -- 172.21.15.50:0/3078384084 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fc26807b970 con 0x7fc27814e470 2024-04-02T16:22:21.359 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:21.360 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc26406c8f0 msgr2=0x7fc26406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc26406c8f0 0x7fc26406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc278069c20 tx=0x7fc274008040 comp rx=0 tx=0).stop 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc27814e470 msgr2=0x7fc2780c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc27814e470 0x7fc2780c84f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc268000c00 tx=0x7fc268001120 comp rx=0 tx=0).stop 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 shutdown_connections 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27806f5a0 0x7fc27806fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc26406c8f0 0x7fc26406edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc27814e470 0x7fc2780c84f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 --2- 172.21.15.50:0/3078384084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc260004800 0x7fc27815b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 >> 172.21.15.50:0/3078384084 conn(0x7fc2780b3b90 msgr2=0x7fc27815fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 shutdown_connections 2024-04-02T16:22:21.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.360+0000 7fc27ed04640 1 -- 172.21.15.50:0/3078384084 wait complete. 2024-04-02T16:22:21.373 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:21.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.477+0000 7f2c40c82640 1 Processor -- start 2024-04-02T16:22:21.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.477+0000 7f2c40c82640 1 -- start start 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.478+0000 7f2c40c82640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f2c3c0d0da0 0x7f2c3c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.478+0000 7f2c40c82640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3c0ccba0 con 0x7f2c3c15c330 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.478+0000 7f2c40c82640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3c0cce40 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.478+0000 7f2c40c82640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3c0cd770 con 0x7f2c3c0d0da0 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.478+0000 7f2c3a59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2c3c15c330 0x7f2c3c0d0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47112/0 (socket says 172.21.15.50:47112) 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.478+0000 7f2c3a59c640 1 -- 172.21.15.50:0/118904147 learned_addr learned my addr 172.21.15.50:0/118904147 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.478+0000 7f2c39d9b640 1 --2- 172.21.15.50:0/118904147 >> v2:172.21.15.50:3300/0 conn(0x7f2c3c0d0da0 0x7f2c3c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.479+0000 7f2c39d9b640 1 -- 172.21.15.50:0/118904147 >> v1:172.21.15.188:6789/0 conn(0x7f2c3c0b46c0 legacy=0x7f2c3c06f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.479+0000 7f2c39d9b640 1 -- 172.21.15.50:0/118904147 >> v1:172.21.15.50:6789/0 conn(0x7f2c3c15c330 legacy=0x7f2c3c0d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.479+0000 7f2c39d9b640 1 -- 172.21.15.50:0/118904147 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c0cd120 con 0x7f2c3c0d0da0 2024-04-02T16:22:21.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.479+0000 7f2c3959a640 1 -- 172.21.15.50:0/118904147 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2452303203 0 0) 0x7f2c3c0ccba0 con 0x7f2c3c15c330 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.480+0000 7f2c39d9b640 1 --2- 172.21.15.50:0/118904147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0d0da0 0x7f2c3c0cdc00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c240097a0 tx=0x7f2c2402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43baef563cb9da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.480+0000 7f2c3959a640 1 -- 172.21.15.50:0/118904147 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24034070 con 0x7f2c3c0d0da0 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.480+0000 7f2c3959a640 1 -- 172.21.15.50:0/118904147 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c24030610 con 0x7f2c3c0d0da0 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.480+0000 7f2c3959a640 1 -- 172.21.15.50:0/118904147 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c240309e0 con 0x7f2c3c0d0da0 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.481+0000 7f2c40c82640 1 -- 172.21.15.50:0/118904147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0d0da0 msgr2=0x7f2c3c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.481+0000 7f2c40c82640 1 --2- 172.21.15.50:0/118904147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0d0da0 0x7f2c3c0cdc00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c240097a0 tx=0x7f2c2402b960 comp rx=0 tx=0).stop 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.481+0000 7f2c40c82640 1 -- 172.21.15.50:0/118904147 shutdown_connections 2024-04-02T16:22:21.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.481+0000 7f2c40c82640 1 --2- 172.21.15.50:0/118904147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0d0da0 0x7f2c3c0cdc00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.481+0000 7f2c40c82640 1 -- 172.21.15.50:0/118904147 >> 172.21.15.50:0/118904147 conn(0x7f2c3c0b3ba0 msgr2=0x7f2c3c0b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:21.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.481+0000 7f2c40c82640 1 -- 172.21.15.50:0/118904147 shutdown_connections 2024-04-02T16:22:21.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.481+0000 7f2c40c82640 1 -- 172.21.15.50:0/118904147 wait complete. 2024-04-02T16:22:21.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.482+0000 7f2c40c82640 1 Processor -- start 2024-04-02T16:22:21.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.482+0000 7f2c40c82640 1 -- start start 2024-04-02T16:22:21.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c40c82640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0b46c0 0x7f2c3c09cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c40c82640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0d0da0 0x7f2c3c09ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c40c82640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15c330 0x7f2c3c0b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c40c82640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0cfe00 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c40c82640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0cfc80 con 0x7f2c3c0d0da0 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c40c82640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c0cff80 con 0x7f2c3c15c330 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0b46c0 0x7f2c3c09cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0b46c0 0x7f2c3c09cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39984/0 (socket says 172.21.15.50:39984) 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 -- 172.21.15.50:0/1703061804 learned_addr learned my addr 172.21.15.50:0/1703061804 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c39d9b640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0d0da0 0x7f2c3c09ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3ad9d640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15c330 0x7f2c3c0b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 -- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15c330 msgr2=0x7f2c3c0b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15c330 0x7f2c3c0b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 -- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0d0da0 msgr2=0x7f2c3c09ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0d0da0 0x7f2c3c09ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.483+0000 7f2c3a59c640 1 -- 172.21.15.50:0/1703061804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c09db70 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.484+0000 7f2c3a59c640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0b46c0 0x7f2c3c09cb70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00c780 tx=0x7f2c2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.484+0000 7f2c32ffd640 1 -- 172.21.15.50:0/1703061804 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c018070 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.484+0000 7f2c32ffd640 1 -- 172.21.15.50:0/1703061804 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c00a640 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.484+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c09c090 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.484+0000 7f2c32ffd640 1 -- 172.21.15.50:0/1703061804 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00aab0 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.484+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c09c650 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.485+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c0756f0 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.485+0000 7f2c32ffd640 1 -- 172.21.15.50:0/1703061804 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c2c00ac50 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.485+0000 7f2c32ffd640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.486+0000 7f2c32ffd640 1 -- 172.21.15.50:0/1703061804 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c2c08b590 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.486+0000 7f2c39d9b640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.486+0000 7f2c39d9b640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c24036000 tx=0x7f2c2403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.490+0000 7f2c32ffd640 1 -- 172.21.15.50:0/1703061804 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2c2c059600 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.676+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c3c0698d0 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.677+0000 7f2c32ffd640 1 -- 172.21.15.50:0/1703061804 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f2c2c058f00 con 0x7f2c3c0b46c0 2024-04-02T16:22:21.679 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:21.679 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2c1c06c8f0 msgr2=0x7f2c1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c24036000 tx=0x7f2c2403b040 comp rx=0 tx=0).stop 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0b46c0 msgr2=0x7f2c3c09cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0b46c0 0x7f2c3c09cb70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00c780 tx=0x7f2c2c00cc40 comp rx=0 tx=0).stop 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 shutdown_connections 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15c330 0x7f2c3c0b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0d0da0 0x7f2c3c09ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.679+0000 7f2c40c82640 1 --2- 172.21.15.50:0/1703061804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c3c0b46c0 0x7f2c3c09cb70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.680+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 >> 172.21.15.50:0/1703061804 conn(0x7f2c3c0b3ba0 msgr2=0x7f2c3c0d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:21.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.680+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 shutdown_connections 2024-04-02T16:22:21.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.680+0000 7f2c40c82640 1 -- 172.21.15.50:0/1703061804 wait complete. 2024-04-02T16:22:21.692 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T16:22:21.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.798+0000 7f7ea22e4640 1 Processor -- start 2024-04-02T16:22:21.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.798+0000 7f7ea22e4640 1 -- start start 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7ea22e4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7e9c160380 0x7f7e9c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7ea22e4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7e9c0b2fe0 0x7f7e9c160760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7ea22e4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e9c0b4a70 con 0x7f7e9c160ca0 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7ea22e4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9c0cfbf0 con 0x7f7e9c160380 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7ea22e4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9c0cd870 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7e9bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7e9c160380 0x7f7e9c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7ea0881640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f7e9c160ca0 0x7f7e9c15f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47126/0 (socket says 172.21.15.50:47126) 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7ea0881640 1 -- 172.21.15.50:0/1644852782 learned_addr learned my addr 172.21.15.50:0/1644852782 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7e9b7fe640 1 --2- 172.21.15.50:0/1644852782 >> v2:172.21.15.188:3300/0 conn(0x7f7e9c0b2fe0 0x7f7e9c160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7e9bfff640 1 -- 172.21.15.50:0/1644852782 >> v2:172.21.15.188:3300/0 conn(0x7f7e9c0b2fe0 msgr2=0x7f7e9c160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:21.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.799+0000 7f7e9bfff640 1 --2- 172.21.15.50:0/1644852782 >> v2:172.21.15.188:3300/0 conn(0x7f7e9c0b2fe0 0x7f7e9c160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.800+0000 7f7e9bfff640 1 -- 172.21.15.50:0/1644852782 >> v1:172.21.15.50:6789/0 conn(0x7f7e9c160ca0 legacy=0x7f7e9c15f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.800+0000 7f7e9bfff640 1 -- 172.21.15.50:0/1644852782 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c0cdb50 con 0x7f7e9c160380 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.800+0000 7f7e9bfff640 1 --2- 172.21.15.50:0/1644852782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c160380 0x7f7e9c0b4e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0727e0 tx=0x7f7e9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7587da0a7dda2756 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.800+0000 7f7e9affd640 1 -- 172.21.15.50:0/1644852782 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e900315a0 con 0x7f7e9c160380 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.800+0000 7f7e9affd640 1 -- 172.21.15.50:0/1644852782 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e90037070 con 0x7f7e9c160380 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.800+0000 7f7e9affd640 1 -- 172.21.15.50:0/1644852782 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90031960 con 0x7f7e9c160380 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1644852782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c160380 msgr2=0x7f7e9c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1644852782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c160380 0x7f7e9c0b4e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0727e0 tx=0x7f7e9002b840 comp rx=0 tx=0).stop 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1644852782 shutdown_connections 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1644852782 >> v2:172.21.15.188:3300/0 conn(0x7f7e9c0b2fe0 0x7f7e9c160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1644852782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c160380 0x7f7e9c0b4e70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1644852782 >> 172.21.15.50:0/1644852782 conn(0x7f7e9c0b3b10 msgr2=0x7f7e9c0b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:21.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1644852782 shutdown_connections 2024-04-02T16:22:21.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.801+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1644852782 wait complete. 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.802+0000 7f7ea22e4640 1 Processor -- start 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7ea22e4640 1 -- start start 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7ea22e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c0b2fe0 0x7f7e9c1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7ea22e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c160380 0x7f7e9c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7ea22e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e9c160ca0 0x7f7e9c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7ea22e4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c15fdd0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7ea22e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c15fc50 con 0x7f7e9c160380 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7ea22e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9c0d0360 con 0x7f7e9c160ca0 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c0b2fe0 0x7f7e9c1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c0b2fe0 0x7f7e9c1506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40004/0 (socket says 172.21.15.50:40004) 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.803+0000 7f7e9bfff640 1 -- 172.21.15.50:0/1563437340 learned_addr learned my addr 172.21.15.50:0/1563437340 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e9b7fe640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c160380 0x7f7e9c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7ea0881640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e9c160ca0 0x7f7e9c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e9bfff640 1 -- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e9c160ca0 msgr2=0x7f7e9c0c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e9bfff640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e9c160ca0 0x7f7e9c0c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e9bfff640 1 -- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c160380 msgr2=0x7f7e9c06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e9bfff640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c160380 0x7f7e9c06cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e9bfff640 1 -- 172.21.15.50:0/1563437340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c06d5e0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e9bfff640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c0b2fe0 0x7f7e9c1506c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009890 tx=0x7f7e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e98ff9640 1 -- 172.21.15.50:0/1563437340 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90032630 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e98ff9640 1 -- 172.21.15.50:0/1563437340 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e900327d0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c06d8c0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7e98ff9640 1 -- 172.21.15.50:0/1563437340 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90032ba0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.804+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c06de80 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.805+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c0756f0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.806+0000 7f7e98ff9640 1 -- 172.21.15.50:0/1563437340 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e9003b3f0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.806+0000 7f7e98ff9640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7e8806c8f0 0x7f7e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:21.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.806+0000 7f7e98ff9640 1 -- 172.21.15.50:0/1563437340 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e9002f030 con 0x7f7e9c0b2fe0 2024-04-02T16:22:21.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.806+0000 7f7e9b7fe640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7e8806c8f0 0x7f7e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:21.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.807+0000 7f7e9b7fe640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7e8806c8f0 0x7f7e8806edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0c98b0 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:21.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:21.811+0000 7f7e98ff9640 1 -- 172.21.15.50:0/1563437340 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7e90040070 con 0x7f7e9c0b2fe0 2024-04-02T16:22:22.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.038+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7e9c0b7fa0 con 0x7f7e9c0b2fe0 2024-04-02T16:22:22.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.472+0000 7f7e98ff9640 1 -- 172.21.15.50:0/1563437340 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f7e9007be10 con 0x7f7e9c0b2fe0 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7e8806c8f0 msgr2=0x7f7e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7e8806c8f0 0x7f7e8806edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0c98b0 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c0b2fe0 msgr2=0x7f7e9c1506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c0b2fe0 0x7f7e9c1506c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009890 tx=0x7f7e900046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 shutdown_connections 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e9c160ca0 0x7f7e9c0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c160380 0x7f7e9c06cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7e8806c8f0 0x7f7e8806edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.474+0000 7f7ea22e4640 1 --2- 172.21.15.50:0/1563437340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e9c0b2fe0 0x7f7e9c1506c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.475+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 >> 172.21.15.50:0/1563437340 conn(0x7f7e9c0b3b10 msgr2=0x7f7e9c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:22.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.475+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 shutdown_connections 2024-04-02T16:22:22.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.475+0000 7f7ea22e4640 1 -- 172.21.15.50:0/1563437340 wait complete. 2024-04-02T16:22:22.490 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-02T16:22:22.490 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-02T16:22:22.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.593+0000 7f36e7c0f640 1 Processor -- start 2024-04-02T16:22:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e7c0f640 1 -- start start 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e7c0f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36e013e170 0x7f36e00b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e7c0f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f36e00b23b0 0x7f36e00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e7c0f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e00b27c0 con 0x7f36e013dd90 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e7c0f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f36e015c3e0 con 0x7f36e00b23b0 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e7c0f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f36e015b570 con 0x7f36e013e170 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e61ac640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f36e00b23b0 0x7f36e00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e61ac640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f36e00b23b0 0x7f36e00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40016/0 (socket says 172.21.15.50:40016) 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.594+0000 7f36e61ac640 1 -- 172.21.15.50:0/3507534422 learned_addr learned my addr 172.21.15.50:0/3507534422 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.595+0000 7f36e51aa640 1 --2- 172.21.15.50:0/3507534422 >> v2:172.21.15.188:3300/0 conn(0x7f36e013e170 0x7f36e00b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.595+0000 7f36e61ac640 1 -- 172.21.15.50:0/3507534422 >> v2:172.21.15.188:3300/0 conn(0x7f36e013e170 msgr2=0x7f36e00b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.595+0000 7f36e61ac640 1 --2- 172.21.15.50:0/3507534422 >> v2:172.21.15.188:3300/0 conn(0x7f36e013e170 0x7f36e00b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.595+0000 7f36e61ac640 1 -- 172.21.15.50:0/3507534422 >> v1:172.21.15.188:6789/0 conn(0x7f36e013dd90 legacy=0x7f36e0141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.595+0000 7f36e61ac640 1 -- 172.21.15.50:0/3507534422 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e015b7d0 con 0x7f36e00b23b0 2024-04-02T16:22:22.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.595+0000 7f36e51aa640 1 --2- 172.21.15.50:0/3507534422 >> v2:172.21.15.188:3300/0 conn(0x7f36e013e170 0x7f36e00b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e61ac640 1 --2- 172.21.15.50:0/3507534422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00cea70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36e00b3540 tx=0x7f36d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3daee672547e461 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e49a9640 1 -- 172.21.15.50:0/3507534422 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d802ca80 con 0x7f36e00b23b0 2024-04-02T16:22:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e49a9640 1 -- 172.21.15.50:0/3507534422 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36d802cc20 con 0x7f36e00b23b0 2024-04-02T16:22:22.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e49a9640 1 -- 172.21.15.50:0/3507534422 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80343f0 con 0x7f36e00b23b0 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 -- 172.21.15.50:0/3507534422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 msgr2=0x7f36e00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/3507534422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00cea70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36e00b3540 tx=0x7f36d802bee0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 -- 172.21.15.50:0/3507534422 shutdown_connections 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/3507534422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00cea70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/3507534422 >> v2:172.21.15.188:3300/0 conn(0x7f36e013e170 0x7f36e00b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 -- 172.21.15.50:0/3507534422 >> 172.21.15.50:0/3507534422 conn(0x7f36e006b700 msgr2=0x7f36e006bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 -- 172.21.15.50:0/3507534422 shutdown_connections 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.596+0000 7f36e7c0f640 1 -- 172.21.15.50:0/3507534422 wait complete. 2024-04-02T16:22:22.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.598+0000 7f36e7c0f640 1 Processor -- start 2024-04-02T16:22:22.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.598+0000 7f36e7c0f640 1 -- start start 2024-04-02T16:22:22.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e7c0f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e7c0f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e013dd90 0x7f36e01f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e7c0f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e013e170 0x7f36e00b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e7c0f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f36e015c070 con 0x7f36e00b23b0 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e7c0f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36e015bd90 con 0x7f36e013dd90 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e7c0f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36e0140890 con 0x7f36e013e170 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40030/0 (socket says 172.21.15.50:40030) 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 -- 172.21.15.50:0/785404560 learned_addr learned my addr 172.21.15.50:0/785404560 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e51aa640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e013dd90 0x7f36e01f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e61ac640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e013e170 0x7f36e00b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 -- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e013e170 msgr2=0x7f36e00b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:22.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e013e170 0x7f36e00b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 -- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e013dd90 msgr2=0x7f36e01f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e013dd90 0x7f36e01f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 -- 172.21.15.50:0/785404560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e01f4fd0 con 0x7f36e00b23b0 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e61ac640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e013e170 0x7f36e00b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e59ab640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00b4c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c430 tx=0x7f36dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36d67fc640 1 -- 172.21.15.50:0/785404560 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc011790 con 0x7f36e00b23b0 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36d67fc640 1 -- 172.21.15.50:0/785404560 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc017070 con 0x7f36e00b23b0 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e01f4210 con 0x7f36e00b23b0 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.599+0000 7f36d67fc640 1 -- 172.21.15.50:0/785404560 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc011bb0 con 0x7f36e00b23b0 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.600+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e01f47d0 con 0x7f36e00b23b0 2024-04-02T16:22:22.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.600+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e01f3f90 con 0x7f36e00b23b0 2024-04-02T16:22:22.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.601+0000 7f36d67fc640 1 -- 172.21.15.50:0/785404560 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36dc01c460 con 0x7f36e00b23b0 2024-04-02T16:22:22.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.601+0000 7f36d67fc640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36c406c920 0x7f36c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:22.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.601+0000 7f36d67fc640 1 -- 172.21.15.50:0/785404560 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f36dc08bca0 con 0x7f36e00b23b0 2024-04-02T16:22:22.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.602+0000 7f36e51aa640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36c406c920 0x7f36c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:22.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.602+0000 7f36e51aa640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36c406c920 0x7f36c406ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36e0155c30 tx=0x7f36d0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:22.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.606+0000 7f36d67fc640 1 -- 172.21.15.50:0/785404560 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f36e01f3f90 con 0x7f36e00b23b0 2024-04-02T16:22:22.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:22.831+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f36e00b3130 con 0x7f36e00b23b0 2024-04-02T16:22:23.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.476+0000 7f36d67fc640 1 -- 172.21.15.50:0/785404560 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f36dc059d90 con 0x7f36e00b23b0 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36c406c920 msgr2=0x7f36c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36c406c920 0x7f36c406ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36e0155c30 tx=0x7f36d0008040 comp rx=0 tx=0).stop 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 msgr2=0x7f36e00b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00b4c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c430 tx=0x7f36dc00c8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 shutdown_connections 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e013e170 0x7f36e00b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e013dd90 0x7f36e01f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36c406c920 0x7f36c406ede0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 --2- 172.21.15.50:0/785404560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36e00b23b0 0x7f36e00b4c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 >> 172.21.15.50:0/785404560 conn(0x7f36e006b700 msgr2=0x7f36e0071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:23.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 shutdown_connections 2024-04-02T16:22:23.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.480+0000 7f36e7c0f640 1 -- 172.21.15.50:0/785404560 wait complete. 2024-04-02T16:22:23.494 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-02T16:22:23.495 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:22:23.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.602+0000 7f17523ba640 1 Processor -- start 2024-04-02T16:22:23.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.602+0000 7f17523ba640 1 -- start start 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f17523ba640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f174c15e650 0x7f174c0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f17523ba640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f174c0d2e80 0x7f174c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f17523ba640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174c0b4c70 con 0x7f174c0b4760 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f17523ba640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f174c0b5880 con 0x7f174c15e650 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f17523ba640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f174c0b4f50 con 0x7f174c0d2e80 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f174bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f174c15e650 0x7f174c0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f174b7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f174c0b4760 0x7f174c0d2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47134/0 (socket says 172.21.15.50:47134) 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f174b7fe640 1 -- 172.21.15.50:0/1292023316 learned_addr learned my addr 172.21.15.50:0/1292023316 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f1750957640 1 --2- 172.21.15.50:0/1292023316 >> v2:172.21.15.188:3300/0 conn(0x7f174c0d2e80 0x7f174c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f174bfff640 1 -- 172.21.15.50:0/1292023316 >> v2:172.21.15.188:3300/0 conn(0x7f174c0d2e80 msgr2=0x7f174c0cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f174bfff640 1 --2- 172.21.15.50:0/1292023316 >> v2:172.21.15.188:3300/0 conn(0x7f174c0d2e80 0x7f174c0cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f174bfff640 1 -- 172.21.15.50:0/1292023316 >> v1:172.21.15.50:6789/0 conn(0x7f174c0b4760 legacy=0x7f174c0d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.603+0000 7f174bfff640 1 -- 172.21.15.50:0/1292023316 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c0b5230 con 0x7f174c15e650 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.604+0000 7f174bfff640 1 --2- 172.21.15.50:0/1292023316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c15e650 0x7f174c0b4220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b5a00 tx=0x7f173402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9f5ceeb2a03da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.604+0000 7f174affd640 1 -- 172.21.15.50:0/1292023316 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734035020 con 0x7f174c15e650 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.604+0000 7f174affd640 1 -- 172.21.15.50:0/1292023316 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1734031830 con 0x7f174c15e650 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.604+0000 7f174affd640 1 -- 172.21.15.50:0/1292023316 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734031c00 con 0x7f174c15e650 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 -- 172.21.15.50:0/1292023316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c15e650 msgr2=0x7f174c0b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 --2- 172.21.15.50:0/1292023316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c15e650 0x7f174c0b4220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b5a00 tx=0x7f173402b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 -- 172.21.15.50:0/1292023316 shutdown_connections 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 --2- 172.21.15.50:0/1292023316 >> v2:172.21.15.188:3300/0 conn(0x7f174c0d2e80 0x7f174c0cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 --2- 172.21.15.50:0/1292023316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c15e650 0x7f174c0b4220 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 -- 172.21.15.50:0/1292023316 >> 172.21.15.50:0/1292023316 conn(0x7f174c09d580 msgr2=0x7f174c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 -- 172.21.15.50:0/1292023316 shutdown_connections 2024-04-02T16:22:23.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.605+0000 7f17523ba640 1 -- 172.21.15.50:0/1292023316 wait complete. 2024-04-02T16:22:23.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.606+0000 7f17523ba640 1 Processor -- start 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.606+0000 7f17523ba640 1 -- start start 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f17523ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f174c0b4760 0x7f174c1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f17523ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c0d2e80 0x7f174c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f17523ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f174c15e650 0x7f174c1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f17523ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f174c15ea30 con 0x7f174c0d2e80 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f17523ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f174c0d2270 con 0x7f174c15e650 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f17523ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f174c15ebb0 con 0x7f174c0b4760 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c0d2e80 0x7f174c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c0d2e80 0x7f174c155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40050/0 (socket says 172.21.15.50:40050) 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 -- 172.21.15.50:0/3083879498 learned_addr learned my addr 172.21.15.50:0/3083879498 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f1750957640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f174c15e650 0x7f174c1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174bfff640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f174c0b4760 0x7f174c1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 -- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f174c0b4760 msgr2=0x7f174c1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f174c0b4760 0x7f174c1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 -- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f174c15e650 msgr2=0x7f174c1f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f174c15e650 0x7f174c1f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.607+0000 7f174b7fe640 1 -- 172.21.15.50:0/3083879498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c1f4cf0 con 0x7f174c0d2e80 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.608+0000 7f174b7fe640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c0d2e80 0x7f174c155af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c440 tx=0x7f173c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.608+0000 7f1748ff9640 1 -- 172.21.15.50:0/3083879498 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c017070 con 0x7f174c0d2e80 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.608+0000 7f1748ff9640 1 -- 172.21.15.50:0/3083879498 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173c005ce0 con 0x7f174c0d2e80 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.608+0000 7f1748ff9640 1 -- 172.21.15.50:0/3083879498 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c0072f0 con 0x7f174c0d2e80 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.608+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c1f4fd0 con 0x7f174c0d2e80 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.608+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c156030 con 0x7f174c0d2e80 2024-04-02T16:22:23.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.609+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c0756f0 con 0x7f174c0d2e80 2024-04-02T16:22:23.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.609+0000 7f1748ff9640 1 -- 172.21.15.50:0/3083879498 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f173c01a070 con 0x7f174c0d2e80 2024-04-02T16:22:23.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.610+0000 7f1748ff9640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f172c06c8f0 0x7f172c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.610+0000 7f1748ff9640 1 -- 172.21.15.50:0/3083879498 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f173c08b090 con 0x7f174c0d2e80 2024-04-02T16:22:23.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.610+0000 7f174bfff640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f172c06c8f0 0x7f172c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.611+0000 7f174bfff640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f172c06c8f0 0x7f172c06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1734036000 tx=0x7f1734008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:23.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.614+0000 7f1748ff9640 1 -- 172.21.15.50:0/3083879498 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f173c058bc0 con 0x7f174c0d2e80 2024-04-02T16:22:23.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.836+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f174c1f51a0 con 0x7f174c0d2e80 2024-04-02T16:22:23.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.836+0000 7f1748ff9640 1 -- 172.21.15.50:0/3083879498 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f173c05cec0 con 0x7f174c0d2e80 2024-04-02T16:22:23.837 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:23.838 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:22:23.838 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:22:23.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f172c06c8f0 msgr2=0x7f172c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f172c06c8f0 0x7f172c06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1734036000 tx=0x7f1734008030 comp rx=0 tx=0).stop 2024-04-02T16:22:23.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c0d2e80 msgr2=0x7f174c155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c0d2e80 0x7f174c155af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c440 tx=0x7f173c00c900 comp rx=0 tx=0).stop 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 shutdown_connections 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f174c15e650 0x7f174c1f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f172c06c8f0 0x7f172c06edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f174c0d2e80 0x7f174c155af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 --2- 172.21.15.50:0/3083879498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f174c0b4760 0x7f174c1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 >> 172.21.15.50:0/3083879498 conn(0x7f174c09d580 msgr2=0x7f174c0cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 shutdown_connections 2024-04-02T16:22:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.838+0000 7f17523ba640 1 -- 172.21.15.50:0/3083879498 wait complete. 2024-04-02T16:22:23.850 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:23.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.950+0000 7f7056406640 1 Processor -- start 2024-04-02T16:22:23.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.950+0000 7f7056406640 1 -- start start 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f7056406640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f705009aa40 0x7f705009ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f7056406640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 0x7f70500cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f7056406640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705009b360 con 0x7f70500b45b0 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f7056406640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f705006f840 con 0x7f705009aa40 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f7056406640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f705006fb20 con 0x7f70500d1320 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704f7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f70500b45b0 0x7f70500cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47138/0 (socket says 172.21.15.50:47138) 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704f7fe640 1 -- 172.21.15.50:0/3994280870 learned_addr learned my addr 172.21.15.50:0/3994280870 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704ffff640 1 --2- 172.21.15.50:0/3994280870 >> v2:172.21.15.188:3300/0 conn(0x7f705009aa40 0x7f705009ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1633689614 0 0) 0x7f705009b360 con 0x7f70500b45b0 2024-04-02T16:22:23.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f70549a3640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 0x7f70500cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f703c003520 con 0x7f70500b45b0 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3903426383 0 0) 0x7f703c003520 con 0x7f70500b45b0 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 msgr2=0x7f70500cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 0x7f70500cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 >> v2:172.21.15.188:3300/0 conn(0x7f705009aa40 msgr2=0x7f705009ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 --2- 172.21.15.50:0/3994280870 >> v2:172.21.15.188:3300/0 conn(0x7f705009aa40 0x7f705009ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500cc5f0 con 0x7f70500b45b0 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.951+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f7040001670 con 0x7f70500b45b0 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 >> v1:172.21.15.50:6789/0 conn(0x7f70500b45b0 legacy=0x7f70500cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704effd640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f703c0039b0 0x7f703c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704effd640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c004800 0x7f703c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704effd640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 0x7f703c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f705006f840 con 0x7f703c0039b0 2024-04-02T16:22:23.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f705006fb20 con 0x7f703c004800 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f703c00b9c0 con 0x7f70500d1320 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704ffff640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f703c0039b0 0x7f703c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704f7fe640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c004800 0x7f703c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f70549a3640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 0x7f703c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704ffff640 1 -- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 msgr2=0x7f703c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704ffff640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 0x7f703c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704ffff640 1 -- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c004800 msgr2=0x7f703c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704ffff640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c004800 0x7f703c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f704ffff640 1 -- 172.21.15.50:0/3994280870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c003520 con 0x7f703c0039b0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.952+0000 7f70549a3640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500d1320 0x7f703c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f704ffff640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f703c0039b0 0x7f703c003dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7038009ac0 tx=0x7f703c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea50d125167a687 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038039070 con 0x7f703c0039b0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703802c710 con 0x7f703c0039b0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f704effd640 1 -- 172.21.15.50:0/3994280870 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038009690 con 0x7f703c0039b0 2024-04-02T16:22:23.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f7056406640 1 -- 172.21.15.50:0/3994280870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f703c0039b0 msgr2=0x7f703c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f7056406640 1 --2- 172.21.15.50:0/3994280870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f703c0039b0 0x7f703c003dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7038009ac0 tx=0x7f703c005830 comp rx=0 tx=0).stop 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f70549a3640 1 -- 172.21.15.50:0/3994280870 reap_dead start 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f7056406640 1 -- 172.21.15.50:0/3994280870 shutdown_connections 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f7056406640 1 -- 172.21.15.50:0/3994280870 >> 172.21.15.50:0/3994280870 conn(0x7f70500b3a90 msgr2=0x7f70500b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f7056406640 1 -- 172.21.15.50:0/3994280870 shutdown_connections 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.953+0000 7f7056406640 1 -- 172.21.15.50:0/3994280870 wait complete. 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.954+0000 7f7056406640 1 Processor -- start 2024-04-02T16:22:23.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f7056406640 1 -- start start 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f7056406640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c0039b0 0x7f70500b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f7056406640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500cbc50 0x7f70501f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f7056406640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70500c97b0 0x7f705015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f7056406640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f705009b860 con 0x7f70500c97b0 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f7056406640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f70500ccbb0 con 0x7f703c0039b0 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f7056406640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f705009b9e0 con 0x7f70500cbc50 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f70549a3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70500c97b0 0x7f705015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f70549a3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70500c97b0 0x7f705015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40066/0 (socket says 172.21.15.50:40066) 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.955+0000 7f70549a3640 1 -- 172.21.15.50:0/3059241590 learned_addr learned my addr 172.21.15.50:0/3059241590 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f704ffff640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c0039b0 0x7f70500b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f704f7fe640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500cbc50 0x7f70501f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f70549a3640 1 -- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500cbc50 msgr2=0x7f70501f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f70549a3640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500cbc50 0x7f70501f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f70549a3640 1 -- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c0039b0 msgr2=0x7f70500b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f70549a3640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c0039b0 0x7f70500b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f70549a3640 1 -- 172.21.15.50:0/3059241590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500cae90 con 0x7f70500c97b0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f704f7fe640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500cbc50 0x7f70501f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f70549a3640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70500c97b0 0x7f705015b690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70500b1670 tx=0x7f7048009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f704cff9640 1 -- 172.21.15.50:0/3059241590 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704801a030 con 0x7f70500c97b0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f704cff9640 1 -- 172.21.15.50:0/3059241590 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7048010e20 con 0x7f70500c97b0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f704cff9640 1 -- 172.21.15.50:0/3059241590 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048003bd0 con 0x7f70500c97b0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70500cb170 con 0x7f70500c97b0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.956+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70500cb730 con 0x7f70500c97b0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.957+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70500cb930 con 0x7f70500c97b0 2024-04-02T16:22:23.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.958+0000 7f704cff9640 1 -- 172.21.15.50:0/3059241590 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f704801d070 con 0x7f70500c97b0 2024-04-02T16:22:23.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.958+0000 7f704cff9640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f703006c8f0 0x7f703006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:23.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.958+0000 7f704cff9640 1 -- 172.21.15.50:0/3059241590 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f704808c970 con 0x7f70500c97b0 2024-04-02T16:22:23.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.958+0000 7f704ffff640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f703006c8f0 0x7f703006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:23.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.959+0000 7f704ffff640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f703006c8f0 0x7f703006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7038036000 tx=0x7f7038030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:23.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:23.962+0000 7f704cff9640 1 -- 172.21.15.50:0/3059241590 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f70500cb930 con 0x7f70500c97b0 2024-04-02T16:22:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.146+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70500ca390 con 0x7f70500c97b0 2024-04-02T16:22:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.147+0000 7f704cff9640 1 -- 172.21.15.50:0/3059241590 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f704805aaf0 con 0x7f70500c97b0 2024-04-02T16:22:24.149 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:24.149 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:24.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f703006c8f0 msgr2=0x7f703006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f703006c8f0 0x7f703006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7038036000 tx=0x7f7038030000 comp rx=0 tx=0).stop 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70500c97b0 msgr2=0x7f705015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70500c97b0 0x7f705015b690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70500b1670 tx=0x7f7048009f90 comp rx=0 tx=0).stop 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 shutdown_connections 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f703006c8f0 0x7f703006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70500c97b0 0x7f705015b690 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70500cbc50 0x7f70501f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 --2- 172.21.15.50:0/3059241590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f703c0039b0 0x7f70500b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 >> 172.21.15.50:0/3059241590 conn(0x7f70500b3a90 msgr2=0x7f70500d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 shutdown_connections 2024-04-02T16:22:24.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.149+0000 7f7056406640 1 -- 172.21.15.50:0/3059241590 wait complete. 2024-04-02T16:22:24.163 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T16:22:24.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:22:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.263+0000 7f393eb9d640 1 Processor -- start 2024-04-02T16:22:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.263+0000 7f393eb9d640 1 -- start start 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393eb9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f393815e370 0x7f39380d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393eb9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f393806d800 0x7f393809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393eb9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39380d12f0 0x7f393809af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393eb9d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f393806d370 con 0x7f393815e370 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393eb9d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f393806c760 con 0x7f393806d800 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393eb9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f393806ca40 con 0x7f39380d12f0 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393c939640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f393815e370 0x7f39380d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393c939640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f393815e370 0x7f39380d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40072/0 (socket says 172.21.15.50:40072) 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.264+0000 7f393c939640 1 -- 172.21.15.50:0/3908907959 learned_addr learned my addr 172.21.15.50:0/3908907959 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f3937fff640 1 --2- 172.21.15.50:0/3908907959 >> v2:172.21.15.188:3300/0 conn(0x7f393806d800 0x7f393809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393d13a640 1 --2- 172.21.15.50:0/3908907959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39380d12f0 0x7f393809af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393c939640 1 -- 172.21.15.50:0/3908907959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39380d12f0 msgr2=0x7f393809af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393c939640 1 --2- 172.21.15.50:0/3908907959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39380d12f0 0x7f393809af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393c939640 1 -- 172.21.15.50:0/3908907959 >> v2:172.21.15.188:3300/0 conn(0x7f393806d800 msgr2=0x7f393809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393c939640 1 --2- 172.21.15.50:0/3908907959 >> v2:172.21.15.188:3300/0 conn(0x7f393806d800 0x7f393809aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393c939640 1 -- 172.21.15.50:0/3908907959 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393806cd20 con 0x7f393815e370 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393d13a640 1 --2- 172.21.15.50:0/3908907959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39380d12f0 0x7f393809af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.265+0000 7f393c939640 1 --2- 172.21.15.50:0/3908907959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f393815e370 0x7f39380d0db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3938072850 tx=0x7f393002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e689dc1930b9d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f39377fe640 1 -- 172.21.15.50:0/3908907959 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930035020 con 0x7f393815e370 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f39377fe640 1 -- 172.21.15.50:0/3908907959 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3930031870 con 0x7f393815e370 2024-04-02T16:22:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f39377fe640 1 -- 172.21.15.50:0/3908907959 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930031c40 con 0x7f393815e370 2024-04-02T16:22:24.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 -- 172.21.15.50:0/3908907959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f393815e370 msgr2=0x7f39380d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 --2- 172.21.15.50:0/3908907959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f393815e370 0x7f39380d0db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3938072850 tx=0x7f393002b910 comp rx=0 tx=0).stop 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 -- 172.21.15.50:0/3908907959 shutdown_connections 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 --2- 172.21.15.50:0/3908907959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39380d12f0 0x7f393809af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 --2- 172.21.15.50:0/3908907959 >> v2:172.21.15.188:3300/0 conn(0x7f393806d800 0x7f393809aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 --2- 172.21.15.50:0/3908907959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f393815e370 0x7f39380d0db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 -- 172.21.15.50:0/3908907959 >> 172.21.15.50:0/3908907959 conn(0x7f39380b3ba0 msgr2=0x7f39380b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 -- 172.21.15.50:0/3908907959 shutdown_connections 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.266+0000 7f393eb9d640 1 -- 172.21.15.50:0/3908907959 wait complete. 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.267+0000 7f393eb9d640 1 Processor -- start 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.267+0000 7f393eb9d640 1 -- start start 2024-04-02T16:22:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393eb9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393806d800 0x7f393814a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393eb9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39380d12f0 0x7f393814b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393eb9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f393815e370 0x7f393814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393eb9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f393815e8d0 con 0x7f39380d12f0 2024-04-02T16:22:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393eb9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f393815e750 con 0x7f393806d800 2024-04-02T16:22:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393eb9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3938162020 con 0x7f393815e370 2024-04-02T16:22:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f3937fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39380d12f0 0x7f393814b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f3937fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39380d12f0 0x7f393814b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40076/0 (socket says 172.21.15.50:40076) 2024-04-02T16:22:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f3937fff640 1 -- 172.21.15.50:0/476019777 learned_addr learned my addr 172.21.15.50:0/476019777 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393d13a640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f393815e370 0x7f393814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.268+0000 7f393c939640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393806d800 0x7f393814a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f3937fff640 1 -- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f393815e370 msgr2=0x7f393814b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f3937fff640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f393815e370 0x7f393814b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f3937fff640 1 -- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393806d800 msgr2=0x7f393814a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f3937fff640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393806d800 0x7f393814a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f3937fff640 1 -- 172.21.15.50:0/476019777 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393806f610 con 0x7f39380d12f0 2024-04-02T16:22:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f3937fff640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39380d12f0 0x7f393814b0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3928007ef0 tx=0x7f392800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f39357fa640 1 -- 172.21.15.50:0/476019777 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928014620 con 0x7f39380d12f0 2024-04-02T16:22:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f39357fa640 1 -- 172.21.15.50:0/476019777 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3928019070 con 0x7f39380d12f0 2024-04-02T16:22:24.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393806f8f0 con 0x7f39380d12f0 2024-04-02T16:22:24.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f39357fa640 1 -- 172.21.15.50:0/476019777 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928014a00 con 0x7f39380d12f0 2024-04-02T16:22:24.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.269+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393806feb0 con 0x7f39380d12f0 2024-04-02T16:22:24.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.270+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39380756f0 con 0x7f39380d12f0 2024-04-02T16:22:24.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.271+0000 7f39357fa640 1 -- 172.21.15.50:0/476019777 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3928014ba0 con 0x7f39380d12f0 2024-04-02T16:22:24.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.271+0000 7f39357fa640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f391806c920 0x7f391806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.271+0000 7f39357fa640 1 -- 172.21.15.50:0/476019777 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f392808c7a0 con 0x7f39380d12f0 2024-04-02T16:22:24.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.271+0000 7f393c939640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f391806c920 0x7f391806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.272+0000 7f393c939640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f391806c920 0x7f391806ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3930036000 tx=0x7f393003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.276+0000 7f39357fa640 1 -- 172.21.15.50:0/476019777 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f392805a890 con 0x7f39380d12f0 2024-04-02T16:22:24.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.457+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 --> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39380b04d0 con 0x7f391806c920 2024-04-02T16:22:24.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.466+0000 7f39357fa640 1 -- 172.21.15.50:0/476019777 <== mgr.4112 v2:172.21.15.188:6832/2785600878 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f39380b04d0 con 0x7f391806c920 2024-04-02T16:22:24.472 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:24.475 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f391806c920 msgr2=0x7f391806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f391806c920 0x7f391806ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3930036000 tx=0x7f393003c040 comp rx=0 tx=0).stop 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39380d12f0 msgr2=0x7f393814b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39380d12f0 0x7f393814b0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3928007ef0 tx=0x7f392800e4b0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 shutdown_connections 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f393815e370 0x7f393814b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f391806c920 0x7f391806ede0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39380d12f0 0x7f393814b0f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 --2- 172.21.15.50:0/476019777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393806d800 0x7f393814a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 >> 172.21.15.50:0/476019777 conn(0x7f39380b3ba0 msgr2=0x7f39380b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:24.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 shutdown_connections 2024-04-02T16:22:24.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.470+0000 7f393eb9d640 1 -- 172.21.15.50:0/476019777 wait complete. 2024-04-02T16:22:24.482 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-02T16:22:22.671209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236104,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738616,"statfs":{"total":773094113280,"available":772852342784,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003074"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434993+0000","last_change":"2024-04-02T16:22:00.392396+0000","last_active":"2024-04-02T16:22:07.434993+0000","last_peered":"2024-04-02T16:22:07.434993+0000","last_clean":"2024-04-02T16:22:07.434993+0000","last_became_active":"2024-04-02T16:22:00.392089+0000","last_became_peered":"2024-04-02T16:22:00.392089+0000","last_unstale":"2024-04-02T16:22:07.434993+0000","last_undegraded":"2024-04-02T16:22:07.434993+0000","last_fullsized":"2024-04-02T16:22:07.434993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428539+0000","last_change":"2024-04-02T16:22:01.410877+0000","last_active":"2024-04-02T16:22:07.428539+0000","last_peered":"2024-04-02T16:22:07.428539+0000","last_clean":"2024-04-02T16:22:07.428539+0000","last_became_active":"2024-04-02T16:22:01.410574+0000","last_became_peered":"2024-04-02T16:22:01.410574+0000","last_unstale":"2024-04-02T16:22:07.428539+0000","last_undegraded":"2024-04-02T16:22:07.428539+0000","last_fullsized":"2024-04-02T16:22:07.428539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427767+0000","last_change":"2024-04-02T16:22:05.817506+0000","last_active":"2024-04-02T16:22:07.427767+0000","last_peered":"2024-04-02T16:22:07.427767+0000","last_clean":"2024-04-02T16:22:07.427767+0000","last_became_active":"2024-04-02T16:22:05.817145+0000","last_became_peered":"2024-04-02T16:22:05.817145+0000","last_unstale":"2024-04-02T16:22:07.427767+0000","last_undegraded":"2024-04-02T16:22:07.427767+0000","last_fullsized":"2024-04-02T16:22:07.427767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:30.473603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428664+0000","last_change":"2024-04-02T16:22:00.391720+0000","last_active":"2024-04-02T16:22:07.428664+0000","last_peered":"2024-04-02T16:22:07.428664+0000","last_clean":"2024-04-02T16:22:07.428664+0000","last_became_active":"2024-04-02T16:22:00.391262+0000","last_became_peered":"2024-04-02T16:22:00.391262+0000","last_unstale":"2024-04-02T16:22:07.428664+0000","last_undegraded":"2024-04-02T16:22:07.428664+0000","last_fullsized":"2024-04-02T16:22:07.428664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434717+0000","last_change":"2024-04-02T16:22:01.403096+0000","last_active":"2024-04-02T16:22:07.434717+0000","last_peered":"2024-04-02T16:22:07.434717+0000","last_clean":"2024-04-02T16:22:07.434717+0000","last_became_active":"2024-04-02T16:22:01.402805+0000","last_became_peered":"2024-04-02T16:22:01.402805+0000","last_unstale":"2024-04-02T16:22:07.434717+0000","last_undegraded":"2024-04-02T16:22:07.434717+0000","last_fullsized":"2024-04-02T16:22:07.434717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428967+0000","last_change":"2024-04-02T16:22:05.447225+0000","last_active":"2024-04-02T16:22:07.428967+0000","last_peered":"2024-04-02T16:22:07.428967+0000","last_clean":"2024-04-02T16:22:07.428967+0000","last_became_active":"2024-04-02T16:22:05.446843+0000","last_became_peered":"2024-04-02T16:22:05.446843+0000","last_unstale":"2024-04-02T16:22:07.428967+0000","last_undegraded":"2024-04-02T16:22:07.428967+0000","last_fullsized":"2024-04-02T16:22:07.428967+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:00.390875+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:00.390501+0000","last_became_peered":"2024-04-02T16:22:00.390501+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430302+0000","last_change":"2024-04-02T16:22:01.410933+0000","last_active":"2024-04-02T16:22:07.430302+0000","last_peered":"2024-04-02T16:22:07.430302+0000","last_clean":"2024-04-02T16:22:07.430302+0000","last_became_active":"2024-04-02T16:22:01.410242+0000","last_became_peered":"2024-04-02T16:22:01.410242+0000","last_unstale":"2024-04-02T16:22:07.430302+0000","last_undegraded":"2024-04-02T16:22:07.430302+0000","last_fullsized":"2024-04-02T16:22:07.430302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429559+0000","last_change":"2024-04-02T16:22:05.440230+0000","last_active":"2024-04-02T16:22:07.429559+0000","last_peered":"2024-04-02T16:22:07.429559+0000","last_clean":"2024-04-02T16:22:07.429559+0000","last_became_active":"2024-04-02T16:22:05.439599+0000","last_became_peered":"2024-04-02T16:22:05.439599+0000","last_unstale":"2024-04-02T16:22:07.429559+0000","last_undegraded":"2024-04-02T16:22:07.429559+0000","last_fullsized":"2024-04-02T16:22:07.429559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:35.350953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434504+0000","last_change":"2024-04-02T16:22:00.392521+0000","last_active":"2024-04-02T16:22:07.434504+0000","last_peered":"2024-04-02T16:22:07.434504+0000","last_clean":"2024-04-02T16:22:07.434504+0000","last_became_active":"2024-04-02T16:22:00.392244+0000","last_became_peered":"2024-04-02T16:22:00.392244+0000","last_unstale":"2024-04-02T16:22:07.434504+0000","last_undegraded":"2024-04-02T16:22:07.434504+0000","last_fullsized":"2024-04-02T16:22:07.434504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430629+0000","last_change":"2024-04-02T16:22:01.406612+0000","last_active":"2024-04-02T16:22:07.430629+0000","last_peered":"2024-04-02T16:22:07.430629+0000","last_clean":"2024-04-02T16:22:07.430629+0000","last_became_active":"2024-04-02T16:22:01.404789+0000","last_became_peered":"2024-04-02T16:22:01.404789+0000","last_unstale":"2024-04-02T16:22:07.430629+0000","last_undegraded":"2024-04-02T16:22:07.430629+0000","last_fullsized":"2024-04-02T16:22:07.430629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429046+0000","last_change":"2024-04-02T16:22:05.445479+0000","last_active":"2024-04-02T16:22:07.429046+0000","last_peered":"2024-04-02T16:22:07.429046+0000","last_clean":"2024-04-02T16:22:07.429046+0000","last_became_active":"2024-04-02T16:22:05.445022+0000","last_became_peered":"2024-04-02T16:22:05.445022+0000","last_unstale":"2024-04-02T16:22:07.429046+0000","last_undegraded":"2024-04-02T16:22:07.429046+0000","last_fullsized":"2024-04-02T16:22:07.429046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.404940+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.404670+0000","last_became_peered":"2024-04-02T16:22:01.404670+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434832+0000","last_change":"2024-04-02T16:22:00.390811+0000","last_active":"2024-04-02T16:22:07.434832+0000","last_peered":"2024-04-02T16:22:07.434832+0000","last_clean":"2024-04-02T16:22:07.434832+0000","last_became_active":"2024-04-02T16:22:00.390515+0000","last_became_peered":"2024-04-02T16:22:00.390515+0000","last_unstale":"2024-04-02T16:22:07.434832+0000","last_undegraded":"2024-04-02T16:22:07.434832+0000","last_fullsized":"2024-04-02T16:22:07.434832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429947+0000","last_change":"2024-04-02T16:22:05.445431+0000","last_active":"2024-04-02T16:22:07.429947+0000","last_peered":"2024-04-02T16:22:07.429947+0000","last_clean":"2024-04-02T16:22:07.429947+0000","last_became_active":"2024-04-02T16:22:05.445014+0000","last_became_peered":"2024-04-02T16:22:05.445014+0000","last_unstale":"2024-04-02T16:22:07.429947+0000","last_undegraded":"2024-04-02T16:22:07.429947+0000","last_fullsized":"2024-04-02T16:22:07.429947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428171+0000","last_change":"2024-04-02T16:22:01.409289+0000","last_active":"2024-04-02T16:22:07.428171+0000","last_peered":"2024-04-02T16:22:07.428171+0000","last_clean":"2024-04-02T16:22:07.428171+0000","last_became_active":"2024-04-02T16:22:01.408934+0000","last_became_peered":"2024-04-02T16:22:01.408934+0000","last_unstale":"2024-04-02T16:22:07.428171+0000","last_undegraded":"2024-04-02T16:22:07.428171+0000","last_fullsized":"2024-04-02T16:22:07.428171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434963+0000","last_change":"2024-04-02T16:22:00.390727+0000","last_active":"2024-04-02T16:22:07.434963+0000","last_peered":"2024-04-02T16:22:07.434963+0000","last_clean":"2024-04-02T16:22:07.434963+0000","last_became_active":"2024-04-02T16:22:00.390131+0000","last_became_peered":"2024-04-02T16:22:00.390131+0000","last_unstale":"2024-04-02T16:22:07.434963+0000","last_undegraded":"2024-04-02T16:22:07.434963+0000","last_fullsized":"2024-04-02T16:22:07.434963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429330+0000","last_change":"2024-04-02T16:22:05.447195+0000","last_active":"2024-04-02T16:22:07.429330+0000","last_peered":"2024-04-02T16:22:07.429330+0000","last_clean":"2024-04-02T16:22:07.429330+0000","last_became_active":"2024-04-02T16:22:05.446753+0000","last_became_peered":"2024-04-02T16:22:05.446753+0000","last_unstale":"2024-04-02T16:22:07.429330+0000","last_undegraded":"2024-04-02T16:22:07.429330+0000","last_fullsized":"2024-04-02T16:22:07.429330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427951+0000","last_change":"2024-04-02T16:22:01.400634+0000","last_active":"2024-04-02T16:22:07.427951+0000","last_peered":"2024-04-02T16:22:07.427951+0000","last_clean":"2024-04-02T16:22:07.427951+0000","last_became_active":"2024-04-02T16:22:01.400309+0000","last_became_peered":"2024-04-02T16:22:01.400309+0000","last_unstale":"2024-04-02T16:22:07.427951+0000","last_undegraded":"2024-04-02T16:22:07.427951+0000","last_fullsized":"2024-04-02T16:22:07.427951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434579+0000","last_change":"2024-04-02T16:22:00.391494+0000","last_active":"2024-04-02T16:22:07.434579+0000","last_peered":"2024-04-02T16:22:07.434579+0000","last_clean":"2024-04-02T16:22:07.434579+0000","last_became_active":"2024-04-02T16:22:00.391190+0000","last_became_peered":"2024-04-02T16:22:00.391190+0000","last_unstale":"2024-04-02T16:22:07.434579+0000","last_undegraded":"2024-04-02T16:22:07.434579+0000","last_fullsized":"2024-04-02T16:22:07.434579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435264+0000","last_change":"2024-04-02T16:22:05.811534+0000","last_active":"2024-04-02T16:22:07.435264+0000","last_peered":"2024-04-02T16:22:07.435264+0000","last_clean":"2024-04-02T16:22:07.435264+0000","last_became_active":"2024-04-02T16:22:05.811134+0000","last_became_peered":"2024-04-02T16:22:05.811134+0000","last_unstale":"2024-04-02T16:22:07.435264+0000","last_undegraded":"2024-04-02T16:22:07.435264+0000","last_fullsized":"2024-04-02T16:22:07.435264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430330+0000","last_change":"2024-04-02T16:22:01.399287+0000","last_active":"2024-04-02T16:22:07.430330+0000","last_peered":"2024-04-02T16:22:07.430330+0000","last_clean":"2024-04-02T16:22:07.430330+0000","last_became_active":"2024-04-02T16:22:01.398956+0000","last_became_peered":"2024-04-02T16:22:01.398956+0000","last_unstale":"2024-04-02T16:22:07.430330+0000","last_undegraded":"2024-04-02T16:22:07.430330+0000","last_fullsized":"2024-04-02T16:22:07.430330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:50.927884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434912+0000","last_change":"2024-04-02T16:22:00.391289+0000","last_active":"2024-04-02T16:22:07.434912+0000","last_peered":"2024-04-02T16:22:07.434912+0000","last_clean":"2024-04-02T16:22:07.434912+0000","last_became_active":"2024-04-02T16:22:00.390292+0000","last_became_peered":"2024-04-02T16:22:00.390292+0000","last_unstale":"2024-04-02T16:22:07.434912+0000","last_undegraded":"2024-04-02T16:22:07.434912+0000","last_fullsized":"2024-04-02T16:22:07.434912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429938+0000","last_change":"2024-04-02T16:22:05.440165+0000","last_active":"2024-04-02T16:22:07.429938+0000","last_peered":"2024-04-02T16:22:07.429938+0000","last_clean":"2024-04-02T16:22:07.429938+0000","last_became_active":"2024-04-02T16:22:05.439732+0000","last_became_peered":"2024-04-02T16:22:05.439732+0000","last_unstale":"2024-04-02T16:22:07.429938+0000","last_undegraded":"2024-04-02T16:22:07.429938+0000","last_fullsized":"2024-04-02T16:22:07.429938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434684+0000","last_change":"2024-04-02T16:22:01.404915+0000","last_active":"2024-04-02T16:22:07.434684+0000","last_peered":"2024-04-02T16:22:07.434684+0000","last_clean":"2024-04-02T16:22:07.434684+0000","last_became_active":"2024-04-02T16:22:01.402769+0000","last_became_peered":"2024-04-02T16:22:01.402769+0000","last_unstale":"2024-04-02T16:22:07.434684+0000","last_undegraded":"2024-04-02T16:22:07.434684+0000","last_fullsized":"2024-04-02T16:22:07.434684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:45.810039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435410+0000","last_change":"2024-04-02T16:22:00.393479+0000","last_active":"2024-04-02T16:22:07.435410+0000","last_peered":"2024-04-02T16:22:07.435410+0000","last_clean":"2024-04-02T16:22:07.435410+0000","last_became_active":"2024-04-02T16:22:00.393178+0000","last_became_peered":"2024-04-02T16:22:00.393178+0000","last_unstale":"2024-04-02T16:22:07.435410+0000","last_undegraded":"2024-04-02T16:22:07.435410+0000","last_fullsized":"2024-04-02T16:22:07.435410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430212+0000","last_change":"2024-04-02T16:22:05.815460+0000","last_active":"2024-04-02T16:22:07.430212+0000","last_peered":"2024-04-02T16:22:07.430212+0000","last_clean":"2024-04-02T16:22:07.430212+0000","last_became_active":"2024-04-02T16:22:05.814988+0000","last_became_peered":"2024-04-02T16:22:05.814988+0000","last_unstale":"2024-04-02T16:22:07.430212+0000","last_undegraded":"2024-04-02T16:22:07.430212+0000","last_fullsized":"2024-04-02T16:22:07.430212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:01.404044+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:01.403758+0000","last_became_peered":"2024-04-02T16:22:01.403758+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435274+0000","last_change":"2024-04-02T16:22:00.394933+0000","last_active":"2024-04-02T16:22:07.435274+0000","last_peered":"2024-04-02T16:22:07.435274+0000","last_clean":"2024-04-02T16:22:07.435274+0000","last_became_active":"2024-04-02T16:22:00.394672+0000","last_became_peered":"2024-04-02T16:22:00.394672+0000","last_unstale":"2024-04-02T16:22:07.435274+0000","last_undegraded":"2024-04-02T16:22:07.435274+0000","last_fullsized":"2024-04-02T16:22:07.435274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427719+0000","last_change":"2024-04-02T16:22:05.816404+0000","last_active":"2024-04-02T16:22:07.427719+0000","last_peered":"2024-04-02T16:22:07.427719+0000","last_clean":"2024-04-02T16:22:07.427719+0000","last_became_active":"2024-04-02T16:22:05.816011+0000","last_became_peered":"2024-04-02T16:22:05.816011+0000","last_unstale":"2024-04-02T16:22:07.427719+0000","last_undegraded":"2024-04-02T16:22:07.427719+0000","last_fullsized":"2024-04-02T16:22:07.427719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:58.796179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:01.405660+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:01.405343+0000","last_became_peered":"2024-04-02T16:22:01.405343+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435169+0000","last_change":"2024-04-02T16:22:00.391246+0000","last_active":"2024-04-02T16:22:07.435169+0000","last_peered":"2024-04-02T16:22:07.435169+0000","last_clean":"2024-04-02T16:22:07.435169+0000","last_became_active":"2024-04-02T16:22:00.390836+0000","last_became_peered":"2024-04-02T16:22:00.390836+0000","last_unstale":"2024-04-02T16:22:07.435169+0000","last_undegraded":"2024-04-02T16:22:07.435169+0000","last_fullsized":"2024-04-02T16:22:07.435169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:22.952130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435359+0000","last_change":"2024-04-02T16:22:05.815150+0000","last_active":"2024-04-02T16:22:07.435359+0000","last_peered":"2024-04-02T16:22:07.435359+0000","last_clean":"2024-04-02T16:22:07.435359+0000","last_became_active":"2024-04-02T16:22:05.814788+0000","last_became_peered":"2024-04-02T16:22:05.814788+0000","last_unstale":"2024-04-02T16:22:07.435359+0000","last_undegraded":"2024-04-02T16:22:07.435359+0000","last_fullsized":"2024-04-02T16:22:07.435359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430743+0000","last_change":"2024-04-02T16:22:01.404482+0000","last_active":"2024-04-02T16:22:07.430743+0000","last_peered":"2024-04-02T16:22:07.430743+0000","last_clean":"2024-04-02T16:22:07.430743+0000","last_became_active":"2024-04-02T16:22:01.402612+0000","last_became_peered":"2024-04-02T16:22:01.402612+0000","last_unstale":"2024-04-02T16:22:07.430743+0000","last_undegraded":"2024-04-02T16:22:07.430743+0000","last_fullsized":"2024-04-02T16:22:07.430743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435056+0000","last_change":"2024-04-02T16:22:00.388850+0000","last_active":"2024-04-02T16:22:07.435056+0000","last_peered":"2024-04-02T16:22:07.435056+0000","last_clean":"2024-04-02T16:22:07.435056+0000","last_became_active":"2024-04-02T16:22:00.388569+0000","last_became_peered":"2024-04-02T16:22:00.388569+0000","last_unstale":"2024-04-02T16:22:07.435056+0000","last_undegraded":"2024-04-02T16:22:07.435056+0000","last_fullsized":"2024-04-02T16:22:07.435056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434575+0000","last_change":"2024-04-02T16:22:05.445952+0000","last_active":"2024-04-02T16:22:07.434575+0000","last_peered":"2024-04-02T16:22:07.434575+0000","last_clean":"2024-04-02T16:22:07.434575+0000","last_became_active":"2024-04-02T16:22:05.445442+0000","last_became_peered":"2024-04-02T16:22:05.445442+0000","last_unstale":"2024-04-02T16:22:07.434575+0000","last_undegraded":"2024-04-02T16:22:07.434575+0000","last_fullsized":"2024-04-02T16:22:07.434575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429867+0000","last_change":"2024-04-02T16:22:01.405033+0000","last_active":"2024-04-02T16:22:07.429867+0000","last_peered":"2024-04-02T16:22:07.429867+0000","last_clean":"2024-04-02T16:22:07.429867+0000","last_became_active":"2024-04-02T16:22:01.404647+0000","last_became_peered":"2024-04-02T16:22:01.404647+0000","last_unstale":"2024-04-02T16:22:07.429867+0000","last_undegraded":"2024-04-02T16:22:07.429867+0000","last_fullsized":"2024-04-02T16:22:07.429867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:03.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429937+0000","last_change":"2024-04-02T16:22:00.390557+0000","last_active":"2024-04-02T16:22:07.429937+0000","last_peered":"2024-04-02T16:22:07.429937+0000","last_clean":"2024-04-02T16:22:07.429937+0000","last_became_active":"2024-04-02T16:22:00.390250+0000","last_became_peered":"2024-04-02T16:22:00.390250+0000","last_unstale":"2024-04-02T16:22:07.429937+0000","last_undegraded":"2024-04-02T16:22:07.429937+0000","last_fullsized":"2024-04-02T16:22:07.429937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434997+0000","last_change":"2024-04-02T16:22:05.448031+0000","last_active":"2024-04-02T16:22:07.434997+0000","last_peered":"2024-04-02T16:22:07.434997+0000","last_clean":"2024-04-02T16:22:07.434997+0000","last_became_active":"2024-04-02T16:22:05.447648+0000","last_became_peered":"2024-04-02T16:22:05.447648+0000","last_unstale":"2024-04-02T16:22:07.434997+0000","last_undegraded":"2024-04-02T16:22:07.434997+0000","last_fullsized":"2024-04-02T16:22:07.434997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433811+0000","last_change":"2024-04-02T16:22:01.406024+0000","last_active":"2024-04-02T16:22:07.433811+0000","last_peered":"2024-04-02T16:22:07.433811+0000","last_clean":"2024-04-02T16:22:07.433811+0000","last_became_active":"2024-04-02T16:22:01.405628+0000","last_became_peered":"2024-04-02T16:22:01.405628+0000","last_unstale":"2024-04-02T16:22:07.433811+0000","last_undegraded":"2024-04-02T16:22:07.433811+0000","last_fullsized":"2024-04-02T16:22:07.433811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433971+0000","last_change":"2024-04-02T16:22:00.384113+0000","last_active":"2024-04-02T16:22:07.433971+0000","last_peered":"2024-04-02T16:22:07.433971+0000","last_clean":"2024-04-02T16:22:07.433971+0000","last_became_active":"2024-04-02T16:22:00.383827+0000","last_became_peered":"2024-04-02T16:22:00.383827+0000","last_unstale":"2024-04-02T16:22:07.433971+0000","last_undegraded":"2024-04-02T16:22:07.433971+0000","last_fullsized":"2024-04-02T16:22:07.433971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435258+0000","last_change":"2024-04-02T16:22:05.448007+0000","last_active":"2024-04-02T16:22:07.435258+0000","last_peered":"2024-04-02T16:22:07.435258+0000","last_clean":"2024-04-02T16:22:07.435258+0000","last_became_active":"2024-04-02T16:22:05.447605+0000","last_became_peered":"2024-04-02T16:22:05.447605+0000","last_unstale":"2024-04-02T16:22:07.435258+0000","last_undegraded":"2024-04-02T16:22:07.435258+0000","last_fullsized":"2024-04-02T16:22:07.435258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434278+0000","last_change":"2024-04-02T16:22:01.394732+0000","last_active":"2024-04-02T16:22:07.434278+0000","last_peered":"2024-04-02T16:22:07.434278+0000","last_clean":"2024-04-02T16:22:07.434278+0000","last_became_active":"2024-04-02T16:22:01.394437+0000","last_became_peered":"2024-04-02T16:22:01.394437+0000","last_unstale":"2024-04-02T16:22:07.434278+0000","last_undegraded":"2024-04-02T16:22:07.434278+0000","last_fullsized":"2024-04-02T16:22:07.434278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:26:16.963906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434233+0000","last_change":"2024-04-02T16:22:00.391466+0000","last_active":"2024-04-02T16:22:07.434233+0000","last_peered":"2024-04-02T16:22:07.434233+0000","last_clean":"2024-04-02T16:22:07.434233+0000","last_became_active":"2024-04-02T16:22:00.391148+0000","last_became_peered":"2024-04-02T16:22:00.391148+0000","last_unstale":"2024-04-02T16:22:07.434233+0000","last_undegraded":"2024-04-02T16:22:07.434233+0000","last_fullsized":"2024-04-02T16:22:07.434233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434509+0000","last_change":"2024-04-02T16:22:05.811459+0000","last_active":"2024-04-02T16:22:07.434509+0000","last_peered":"2024-04-02T16:22:07.434509+0000","last_clean":"2024-04-02T16:22:07.434509+0000","last_became_active":"2024-04-02T16:22:05.811036+0000","last_became_peered":"2024-04-02T16:22:05.811036+0000","last_unstale":"2024-04-02T16:22:07.434509+0000","last_undegraded":"2024-04-02T16:22:07.434509+0000","last_fullsized":"2024-04-02T16:22:07.434509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428029+0000","last_change":"2024-04-02T16:22:01.410992+0000","last_active":"2024-04-02T16:22:07.428029+0000","last_peered":"2024-04-02T16:22:07.428029+0000","last_clean":"2024-04-02T16:22:07.428029+0000","last_became_active":"2024-04-02T16:22:01.410701+0000","last_became_peered":"2024-04-02T16:22:01.410701+0000","last_unstale":"2024-04-02T16:22:07.428029+0000","last_undegraded":"2024-04-02T16:22:07.428029+0000","last_fullsized":"2024-04-02T16:22:07.428029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429896+0000","last_change":"2024-04-02T16:22:00.382563+0000","last_active":"2024-04-02T16:22:07.429896+0000","last_peered":"2024-04-02T16:22:07.429896+0000","last_clean":"2024-04-02T16:22:07.429896+0000","last_became_active":"2024-04-02T16:22:00.382285+0000","last_became_peered":"2024-04-02T16:22:00.382285+0000","last_unstale":"2024-04-02T16:22:07.429896+0000","last_undegraded":"2024-04-02T16:22:07.429896+0000","last_fullsized":"2024-04-02T16:22:07.429896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:21:49.851609+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:21:49.850564+0000","last_became_peered":"2024-04-02T16:21:49.850564+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_clean_scrub_stamp":"2024-04-02T16:21:48.692321+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.825754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434765+0000","last_change":"2024-04-02T16:22:05.816624+0000","last_active":"2024-04-02T16:22:07.434765+0000","last_peered":"2024-04-02T16:22:07.434765+0000","last_clean":"2024-04-02T16:22:07.434765+0000","last_became_active":"2024-04-02T16:22:05.816230+0000","last_became_peered":"2024-04-02T16:22:05.816230+0000","last_unstale":"2024-04-02T16:22:07.434765+0000","last_undegraded":"2024-04-02T16:22:07.434765+0000","last_fullsized":"2024-04-02T16:22:07.434765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434871+0000","last_change":"2024-04-02T16:22:01.412438+0000","last_active":"2024-04-02T16:22:07.434871+0000","last_peered":"2024-04-02T16:22:07.434871+0000","last_clean":"2024-04-02T16:22:07.434871+0000","last_became_active":"2024-04-02T16:22:01.411845+0000","last_became_peered":"2024-04-02T16:22:01.411845+0000","last_unstale":"2024-04-02T16:22:07.434871+0000","last_undegraded":"2024-04-02T16:22:07.434871+0000","last_fullsized":"2024-04-02T16:22:07.434871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:22:00.381230+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:22:00.380907+0000","last_became_peered":"2024-04-02T16:22:00.380907+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434320+0000","last_change":"2024-04-02T16:22:05.815548+0000","last_active":"2024-04-02T16:22:07.434320+0000","last_peered":"2024-04-02T16:22:07.434320+0000","last_clean":"2024-04-02T16:22:07.434320+0000","last_became_active":"2024-04-02T16:22:05.815073+0000","last_became_peered":"2024-04-02T16:22:05.815073+0000","last_unstale":"2024-04-02T16:22:07.434320+0000","last_undegraded":"2024-04-02T16:22:07.434320+0000","last_fullsized":"2024-04-02T16:22:07.434320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430426+0000","last_change":"2024-04-02T16:22:01.407537+0000","last_active":"2024-04-02T16:22:07.430426+0000","last_peered":"2024-04-02T16:22:07.430426+0000","last_clean":"2024-04-02T16:22:07.430426+0000","last_became_active":"2024-04-02T16:22:01.403892+0000","last_became_peered":"2024-04-02T16:22:01.403892+0000","last_unstale":"2024-04-02T16:22:07.430426+0000","last_undegraded":"2024-04-02T16:22:07.430426+0000","last_fullsized":"2024-04-02T16:22:07.430426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434911+0000","last_change":"2024-04-02T16:22:00.387875+0000","last_active":"2024-04-02T16:22:07.434911+0000","last_peered":"2024-04-02T16:22:07.434911+0000","last_clean":"2024-04-02T16:22:07.434911+0000","last_became_active":"2024-04-02T16:22:00.387562+0000","last_became_peered":"2024-04-02T16:22:00.387562+0000","last_unstale":"2024-04-02T16:22:07.434911+0000","last_undegraded":"2024-04-02T16:22:07.434911+0000","last_fullsized":"2024-04-02T16:22:07.434911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435508+0000","last_change":"2024-04-02T16:22:05.450999+0000","last_active":"2024-04-02T16:22:07.435508+0000","last_peered":"2024-04-02T16:22:07.435508+0000","last_clean":"2024-04-02T16:22:07.435508+0000","last_became_active":"2024-04-02T16:22:05.450407+0000","last_became_peered":"2024-04-02T16:22:05.450407+0000","last_unstale":"2024-04-02T16:22:07.435508+0000","last_undegraded":"2024-04-02T16:22:07.435508+0000","last_fullsized":"2024-04-02T16:22:07.435508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435483+0000","last_change":"2024-04-02T16:22:01.410592+0000","last_active":"2024-04-02T16:22:07.435483+0000","last_peered":"2024-04-02T16:22:07.435483+0000","last_clean":"2024-04-02T16:22:07.435483+0000","last_became_active":"2024-04-02T16:22:01.410288+0000","last_became_peered":"2024-04-02T16:22:01.410288+0000","last_unstale":"2024-04-02T16:22:07.435483+0000","last_undegraded":"2024-04-02T16:22:07.435483+0000","last_fullsized":"2024-04-02T16:22:07.435483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:58.411440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434019+0000","last_change":"2024-04-02T16:22:00.389059+0000","last_active":"2024-04-02T16:22:07.434019+0000","last_peered":"2024-04-02T16:22:07.434019+0000","last_clean":"2024-04-02T16:22:07.434019+0000","last_became_active":"2024-04-02T16:22:00.388780+0000","last_became_peered":"2024-04-02T16:22:00.388780+0000","last_unstale":"2024-04-02T16:22:07.434019+0000","last_undegraded":"2024-04-02T16:22:07.434019+0000","last_fullsized":"2024-04-02T16:22:07.434019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430338+0000","last_change":"2024-04-02T16:22:05.811513+0000","last_active":"2024-04-02T16:22:07.430338+0000","last_peered":"2024-04-02T16:22:07.430338+0000","last_clean":"2024-04-02T16:22:07.430338+0000","last_became_active":"2024-04-02T16:22:05.811011+0000","last_became_peered":"2024-04-02T16:22:05.811011+0000","last_unstale":"2024-04-02T16:22:07.430338+0000","last_undegraded":"2024-04-02T16:22:07.430338+0000","last_fullsized":"2024-04-02T16:22:07.430338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435147+0000","last_change":"2024-04-02T16:22:01.404383+0000","last_active":"2024-04-02T16:22:07.435147+0000","last_peered":"2024-04-02T16:22:07.435147+0000","last_clean":"2024-04-02T16:22:07.435147+0000","last_became_active":"2024-04-02T16:22:01.404105+0000","last_became_peered":"2024-04-02T16:22:01.404105+0000","last_unstale":"2024-04-02T16:22:07.435147+0000","last_undegraded":"2024-04-02T16:22:07.435147+0000","last_fullsized":"2024-04-02T16:22:07.435147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429344+0000","last_change":"2024-04-02T16:22:00.385712+0000","last_active":"2024-04-02T16:22:07.429344+0000","last_peered":"2024-04-02T16:22:07.429344+0000","last_clean":"2024-04-02T16:22:07.429344+0000","last_became_active":"2024-04-02T16:22:00.385374+0000","last_became_peered":"2024-04-02T16:22:00.385374+0000","last_unstale":"2024-04-02T16:22:07.429344+0000","last_undegraded":"2024-04-02T16:22:07.429344+0000","last_fullsized":"2024-04-02T16:22:07.429344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430015+0000","last_change":"2024-04-02T16:22:05.439393+0000","last_active":"2024-04-02T16:22:07.430015+0000","last_peered":"2024-04-02T16:22:07.430015+0000","last_clean":"2024-04-02T16:22:07.430015+0000","last_became_active":"2024-04-02T16:22:05.439022+0000","last_became_peered":"2024-04-02T16:22:05.439022+0000","last_unstale":"2024-04-02T16:22:07.430015+0000","last_undegraded":"2024-04-02T16:22:07.430015+0000","last_fullsized":"2024-04-02T16:22:07.430015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434659+0000","last_change":"2024-04-02T16:22:01.412452+0000","last_active":"2024-04-02T16:22:07.434659+0000","last_peered":"2024-04-02T16:22:07.434659+0000","last_clean":"2024-04-02T16:22:07.434659+0000","last_became_active":"2024-04-02T16:22:01.412160+0000","last_became_peered":"2024-04-02T16:22:01.412160+0000","last_unstale":"2024-04-02T16:22:07.434659+0000","last_undegraded":"2024-04-02T16:22:07.434659+0000","last_fullsized":"2024-04-02T16:22:07.434659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433532+0000","last_change":"2024-04-02T16:22:00.386167+0000","last_active":"2024-04-02T16:22:07.433532+0000","last_peered":"2024-04-02T16:22:07.433532+0000","last_clean":"2024-04-02T16:22:07.433532+0000","last_became_active":"2024-04-02T16:22:00.385887+0000","last_became_peered":"2024-04-02T16:22:00.385887+0000","last_unstale":"2024-04-02T16:22:07.433532+0000","last_undegraded":"2024-04-02T16:22:07.433532+0000","last_fullsized":"2024-04-02T16:22:07.433532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434360+0000","last_change":"2024-04-02T16:22:05.451144+0000","last_active":"2024-04-02T16:22:07.434360+0000","last_peered":"2024-04-02T16:22:07.434360+0000","last_clean":"2024-04-02T16:22:07.434360+0000","last_became_active":"2024-04-02T16:22:05.450690+0000","last_became_peered":"2024-04-02T16:22:05.450690+0000","last_unstale":"2024-04-02T16:22:07.434360+0000","last_undegraded":"2024-04-02T16:22:07.434360+0000","last_fullsized":"2024-04-02T16:22:07.434360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434308+0000","last_change":"2024-04-02T16:22:01.409326+0000","last_active":"2024-04-02T16:22:07.434308+0000","last_peered":"2024-04-02T16:22:07.434308+0000","last_clean":"2024-04-02T16:22:07.434308+0000","last_became_active":"2024-04-02T16:22:01.408710+0000","last_became_peered":"2024-04-02T16:22:01.408710+0000","last_unstale":"2024-04-02T16:22:07.434308+0000","last_undegraded":"2024-04-02T16:22:07.434308+0000","last_fullsized":"2024-04-02T16:22:07.434308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434473+0000","last_change":"2024-04-02T16:22:00.389406+0000","last_active":"2024-04-02T16:22:07.434473+0000","last_peered":"2024-04-02T16:22:07.434473+0000","last_clean":"2024-04-02T16:22:07.434473+0000","last_became_active":"2024-04-02T16:22:00.389107+0000","last_became_peered":"2024-04-02T16:22:00.389107+0000","last_unstale":"2024-04-02T16:22:07.434473+0000","last_undegraded":"2024-04-02T16:22:07.434473+0000","last_fullsized":"2024-04-02T16:22:07.434473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:05.816005+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:45.290851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434745+0000","last_change":"2024-04-02T16:22:01.412447+0000","last_active":"2024-04-02T16:22:07.434745+0000","last_peered":"2024-04-02T16:22:07.434745+0000","last_clean":"2024-04-02T16:22:07.434745+0000","last_became_active":"2024-04-02T16:22:01.411677+0000","last_became_peered":"2024-04-02T16:22:01.411677+0000","last_unstale":"2024-04-02T16:22:07.434745+0000","last_undegraded":"2024-04-02T16:22:07.434745+0000","last_fullsized":"2024-04-02T16:22:07.434745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:00.389887+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:00.389287+0000","last_became_peered":"2024-04-02T16:22:00.389287+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428399+0000","last_change":"2024-04-02T16:22:05.815236+0000","last_active":"2024-04-02T16:22:07.428399+0000","last_peered":"2024-04-02T16:22:07.428399+0000","last_clean":"2024-04-02T16:22:07.428399+0000","last_became_active":"2024-04-02T16:22:05.814782+0000","last_became_peered":"2024-04-02T16:22:05.814782+0000","last_unstale":"2024-04-02T16:22:07.428399+0000","last_undegraded":"2024-04-02T16:22:07.428399+0000","last_fullsized":"2024-04-02T16:22:07.428399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:44.247231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434209+0000","last_change":"2024-04-02T16:22:01.401164+0000","last_active":"2024-04-02T16:22:07.434209+0000","last_peered":"2024-04-02T16:22:07.434209+0000","last_clean":"2024-04-02T16:22:07.434209+0000","last_became_active":"2024-04-02T16:22:01.400880+0000","last_became_peered":"2024-04-02T16:22:01.400880+0000","last_unstale":"2024-04-02T16:22:07.434209+0000","last_undegraded":"2024-04-02T16:22:07.434209+0000","last_fullsized":"2024-04-02T16:22:07.434209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435570+0000","last_change":"2024-04-02T16:22:00.386660+0000","last_active":"2024-04-02T16:22:07.435570+0000","last_peered":"2024-04-02T16:22:07.435570+0000","last_clean":"2024-04-02T16:22:07.435570+0000","last_became_active":"2024-04-02T16:22:00.386370+0000","last_became_peered":"2024-04-02T16:22:00.386370+0000","last_unstale":"2024-04-02T16:22:07.435570+0000","last_undegraded":"2024-04-02T16:22:07.435570+0000","last_fullsized":"2024-04-02T16:22:07.435570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430760+0000","last_change":"2024-04-02T16:22:05.816956+0000","last_active":"2024-04-02T16:22:07.430760+0000","last_peered":"2024-04-02T16:22:07.430760+0000","last_clean":"2024-04-02T16:22:07.430760+0000","last_became_active":"2024-04-02T16:22:05.816429+0000","last_became_peered":"2024-04-02T16:22:05.816429+0000","last_unstale":"2024-04-02T16:22:07.430760+0000","last_undegraded":"2024-04-02T16:22:07.430760+0000","last_fullsized":"2024-04-02T16:22:07.430760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434384+0000","last_change":"2024-04-02T16:22:01.405624+0000","last_active":"2024-04-02T16:22:07.434384+0000","last_peered":"2024-04-02T16:22:07.434384+0000","last_clean":"2024-04-02T16:22:07.434384+0000","last_became_active":"2024-04-02T16:22:01.405347+0000","last_became_peered":"2024-04-02T16:22:01.405347+0000","last_unstale":"2024-04-02T16:22:07.434384+0000","last_undegraded":"2024-04-02T16:22:07.434384+0000","last_fullsized":"2024-04-02T16:22:07.434384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:10.942808+0000","last_change":"2024-04-02T16:22:00.391986+0000","last_active":"2024-04-02T16:22:10.942808+0000","last_peered":"2024-04-02T16:22:10.942808+0000","last_clean":"2024-04-02T16:22:10.942808+0000","last_became_active":"2024-04-02T16:22:00.391681+0000","last_became_peered":"2024-04-02T16:22:00.391681+0000","last_unstale":"2024-04-02T16:22:10.942808+0000","last_undegraded":"2024-04-02T16:22:10.942808+0000","last_fullsized":"2024-04-02T16:22:10.942808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434848+0000","last_change":"2024-04-02T16:22:05.446704+0000","last_active":"2024-04-02T16:22:07.434848+0000","last_peered":"2024-04-02T16:22:07.434848+0000","last_clean":"2024-04-02T16:22:07.434848+0000","last_became_active":"2024-04-02T16:22:05.446150+0000","last_became_peered":"2024-04-02T16:22:05.446150+0000","last_unstale":"2024-04-02T16:22:07.434848+0000","last_undegraded":"2024-04-02T16:22:07.434848+0000","last_fullsized":"2024-04-02T16:22:07.434848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:01.400842+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:01.400565+0000","last_became_peered":"2024-04-02T16:22:01.400565+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434908+0000","last_change":"2024-04-02T16:22:00.394135+0000","last_active":"2024-04-02T16:22:07.434908+0000","last_peered":"2024-04-02T16:22:07.434908+0000","last_clean":"2024-04-02T16:22:07.434908+0000","last_became_active":"2024-04-02T16:22:00.393846+0000","last_became_peered":"2024-04-02T16:22:00.393846+0000","last_unstale":"2024-04-02T16:22:07.434908+0000","last_undegraded":"2024-04-02T16:22:07.434908+0000","last_fullsized":"2024-04-02T16:22:07.434908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429907+0000","last_change":"2024-04-02T16:22:05.439118+0000","last_active":"2024-04-02T16:22:07.429907+0000","last_peered":"2024-04-02T16:22:07.429907+0000","last_clean":"2024-04-02T16:22:07.429907+0000","last_became_active":"2024-04-02T16:22:05.438557+0000","last_became_peered":"2024-04-02T16:22:05.438557+0000","last_unstale":"2024-04-02T16:22:07.429907+0000","last_undegraded":"2024-04-02T16:22:07.429907+0000","last_fullsized":"2024-04-02T16:22:07.429907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434885+0000","last_change":"2024-04-02T16:22:01.413531+0000","last_active":"2024-04-02T16:22:07.434885+0000","last_peered":"2024-04-02T16:22:07.434885+0000","last_clean":"2024-04-02T16:22:07.434885+0000","last_became_active":"2024-04-02T16:22:01.413229+0000","last_became_peered":"2024-04-02T16:22:01.413229+0000","last_unstale":"2024-04-02T16:22:07.434885+0000","last_undegraded":"2024-04-02T16:22:07.434885+0000","last_fullsized":"2024-04-02T16:22:07.434885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430736+0000","last_change":"2024-04-02T16:22:00.388772+0000","last_active":"2024-04-02T16:22:07.430736+0000","last_peered":"2024-04-02T16:22:07.430736+0000","last_clean":"2024-04-02T16:22:07.430736+0000","last_became_active":"2024-04-02T16:22:00.388500+0000","last_became_peered":"2024-04-02T16:22:00.388500+0000","last_unstale":"2024-04-02T16:22:07.430736+0000","last_undegraded":"2024-04-02T16:22:07.430736+0000","last_fullsized":"2024-04-02T16:22:07.430736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427878+0000","last_change":"2024-04-02T16:22:05.441542+0000","last_active":"2024-04-02T16:22:07.427878+0000","last_peered":"2024-04-02T16:22:07.427878+0000","last_clean":"2024-04-02T16:22:07.427878+0000","last_became_active":"2024-04-02T16:22:05.441134+0000","last_became_peered":"2024-04-02T16:22:05.441134+0000","last_unstale":"2024-04-02T16:22:07.427878+0000","last_undegraded":"2024-04-02T16:22:07.427878+0000","last_fullsized":"2024-04-02T16:22:07.427878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:02:44.237629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430701+0000","last_change":"2024-04-02T16:22:01.401315+0000","last_active":"2024-04-02T16:22:07.430701+0000","last_peered":"2024-04-02T16:22:07.430701+0000","last_clean":"2024-04-02T16:22:07.430701+0000","last_became_active":"2024-04-02T16:22:01.401023+0000","last_became_peered":"2024-04-02T16:22:01.401023+0000","last_unstale":"2024-04-02T16:22:07.430701+0000","last_undegraded":"2024-04-02T16:22:07.430701+0000","last_fullsized":"2024-04-02T16:22:07.430701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435375+0000","last_change":"2024-04-02T16:22:00.394082+0000","last_active":"2024-04-02T16:22:07.435375+0000","last_peered":"2024-04-02T16:22:07.435375+0000","last_clean":"2024-04-02T16:22:07.435375+0000","last_became_active":"2024-04-02T16:22:00.393793+0000","last_became_peered":"2024-04-02T16:22:00.393793+0000","last_unstale":"2024-04-02T16:22:07.435375+0000","last_undegraded":"2024-04-02T16:22:07.435375+0000","last_fullsized":"2024-04-02T16:22:07.435375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427851+0000","last_change":"2024-04-02T16:22:05.817432+0000","last_active":"2024-04-02T16:22:07.427851+0000","last_peered":"2024-04-02T16:22:07.427851+0000","last_clean":"2024-04-02T16:22:07.427851+0000","last_became_active":"2024-04-02T16:22:05.817018+0000","last_became_peered":"2024-04-02T16:22:05.817018+0000","last_unstale":"2024-04-02T16:22:07.427851+0000","last_undegraded":"2024-04-02T16:22:07.427851+0000","last_fullsized":"2024-04-02T16:22:07.427851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:52:22.886852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434205+0000","last_change":"2024-04-02T16:22:05.815342+0000","last_active":"2024-04-02T16:22:07.434205+0000","last_peered":"2024-04-02T16:22:07.434205+0000","last_clean":"2024-04-02T16:22:07.434205+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.434205+0000","last_undegraded":"2024-04-02T16:22:07.434205+0000","last_fullsized":"2024-04-02T16:22:07.434205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434398+0000","last_change":"2024-04-02T16:22:01.400394+0000","last_active":"2024-04-02T16:22:07.434398+0000","last_peered":"2024-04-02T16:22:07.434398+0000","last_clean":"2024-04-02T16:22:07.434398+0000","last_became_active":"2024-04-02T16:22:01.400000+0000","last_became_peered":"2024-04-02T16:22:01.400000+0000","last_unstale":"2024-04-02T16:22:07.434398+0000","last_undegraded":"2024-04-02T16:22:07.434398+0000","last_fullsized":"2024-04-02T16:22:07.434398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430602+0000","last_change":"2024-04-02T16:22:00.380328+0000","last_active":"2024-04-02T16:22:07.430602+0000","last_peered":"2024-04-02T16:22:07.430602+0000","last_clean":"2024-04-02T16:22:07.430602+0000","last_became_active":"2024-04-02T16:22:00.379294+0000","last_became_peered":"2024-04-02T16:22:00.379294+0000","last_unstale":"2024-04-02T16:22:07.430602+0000","last_undegraded":"2024-04-02T16:22:07.430602+0000","last_fullsized":"2024-04-02T16:22:07.430602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435187+0000","last_change":"2024-04-02T16:22:05.444408+0000","last_active":"2024-04-02T16:22:07.435187+0000","last_peered":"2024-04-02T16:22:07.435187+0000","last_clean":"2024-04-02T16:22:07.435187+0000","last_became_active":"2024-04-02T16:22:05.444100+0000","last_became_peered":"2024-04-02T16:22:05.444100+0000","last_unstale":"2024-04-02T16:22:07.435187+0000","last_undegraded":"2024-04-02T16:22:07.435187+0000","last_fullsized":"2024-04-02T16:22:07.435187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427872+0000","last_change":"2024-04-02T16:22:01.409456+0000","last_active":"2024-04-02T16:22:07.427872+0000","last_peered":"2024-04-02T16:22:07.427872+0000","last_clean":"2024-04-02T16:22:07.427872+0000","last_became_active":"2024-04-02T16:22:01.408824+0000","last_became_peered":"2024-04-02T16:22:01.408824+0000","last_unstale":"2024-04-02T16:22:07.427872+0000","last_undegraded":"2024-04-02T16:22:07.427872+0000","last_fullsized":"2024-04-02T16:22:07.427872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427970+0000","last_change":"2024-04-02T16:22:00.390487+0000","last_active":"2024-04-02T16:22:07.427970+0000","last_peered":"2024-04-02T16:22:07.427970+0000","last_clean":"2024-04-02T16:22:07.427970+0000","last_became_active":"2024-04-02T16:22:00.390198+0000","last_became_peered":"2024-04-02T16:22:00.390198+0000","last_unstale":"2024-04-02T16:22:07.427970+0000","last_undegraded":"2024-04-02T16:22:07.427970+0000","last_fullsized":"2024-04-02T16:22:07.427970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434683+0000","last_change":"2024-04-02T16:22:00.388624+0000","last_active":"2024-04-02T16:22:07.434683+0000","last_peered":"2024-04-02T16:22:07.434683+0000","last_clean":"2024-04-02T16:22:07.434683+0000","last_became_active":"2024-04-02T16:22:00.388351+0000","last_became_peered":"2024-04-02T16:22:00.388351+0000","last_unstale":"2024-04-02T16:22:07.434683+0000","last_undegraded":"2024-04-02T16:22:07.434683+0000","last_fullsized":"2024-04-02T16:22:07.434683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435511+0000","last_change":"2024-04-02T16:22:01.403982+0000","last_active":"2024-04-02T16:22:07.435511+0000","last_peered":"2024-04-02T16:22:07.435511+0000","last_clean":"2024-04-02T16:22:07.435511+0000","last_became_active":"2024-04-02T16:22:01.403057+0000","last_became_peered":"2024-04-02T16:22:01.403057+0000","last_unstale":"2024-04-02T16:22:07.435511+0000","last_undegraded":"2024-04-02T16:22:07.435511+0000","last_fullsized":"2024-04-02T16:22:07.435511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434167+0000","last_change":"2024-04-02T16:22:05.443373+0000","last_active":"2024-04-02T16:22:07.434167+0000","last_peered":"2024-04-02T16:22:07.434167+0000","last_clean":"2024-04-02T16:22:07.434167+0000","last_became_active":"2024-04-02T16:22:05.442934+0000","last_became_peered":"2024-04-02T16:22:05.442934+0000","last_unstale":"2024-04-02T16:22:07.434167+0000","last_undegraded":"2024-04-02T16:22:07.434167+0000","last_fullsized":"2024-04-02T16:22:07.434167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:49.857086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428264+0000","last_change":"2024-04-02T16:22:00.391680+0000","last_active":"2024-04-02T16:22:07.428264+0000","last_peered":"2024-04-02T16:22:07.428264+0000","last_clean":"2024-04-02T16:22:07.428264+0000","last_became_active":"2024-04-02T16:22:00.391402+0000","last_became_peered":"2024-04-02T16:22:00.391402+0000","last_unstale":"2024-04-02T16:22:07.428264+0000","last_undegraded":"2024-04-02T16:22:07.428264+0000","last_fullsized":"2024-04-02T16:22:07.428264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429224+0000","last_change":"2024-04-02T16:22:01.398824+0000","last_active":"2024-04-02T16:22:07.429224+0000","last_peered":"2024-04-02T16:22:07.429224+0000","last_clean":"2024-04-02T16:22:07.429224+0000","last_became_active":"2024-04-02T16:22:01.398529+0000","last_became_peered":"2024-04-02T16:22:01.398529+0000","last_unstale":"2024-04-02T16:22:07.429224+0000","last_undegraded":"2024-04-02T16:22:07.429224+0000","last_fullsized":"2024-04-02T16:22:07.429224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:01.320427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428686+0000","last_change":"2024-04-02T16:22:05.440423+0000","last_active":"2024-04-02T16:22:07.428686+0000","last_peered":"2024-04-02T16:22:07.428686+0000","last_clean":"2024-04-02T16:22:07.428686+0000","last_became_active":"2024-04-02T16:22:05.439993+0000","last_became_peered":"2024-04-02T16:22:05.439993+0000","last_unstale":"2024-04-02T16:22:07.428686+0000","last_undegraded":"2024-04-02T16:22:07.428686+0000","last_fullsized":"2024-04-02T16:22:07.428686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:17.805386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434177+0000","last_change":"2024-04-02T16:22:01.408902+0000","last_active":"2024-04-02T16:22:07.434177+0000","last_peered":"2024-04-02T16:22:07.434177+0000","last_clean":"2024-04-02T16:22:07.434177+0000","last_became_active":"2024-04-02T16:22:01.408562+0000","last_became_peered":"2024-04-02T16:22:01.408562+0000","last_unstale":"2024-04-02T16:22:07.434177+0000","last_undegraded":"2024-04-02T16:22:07.434177+0000","last_fullsized":"2024-04-02T16:22:07.434177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433873+0000","last_change":"2024-04-02T16:22:01.404001+0000","last_active":"2024-04-02T16:22:07.433873+0000","last_peered":"2024-04-02T16:22:07.433873+0000","last_clean":"2024-04-02T16:22:07.433873+0000","last_became_active":"2024-04-02T16:22:01.403694+0000","last_became_peered":"2024-04-02T16:22:01.403694+0000","last_unstale":"2024-04-02T16:22:07.433873+0000","last_undegraded":"2024-04-02T16:22:07.433873+0000","last_fullsized":"2024-04-02T16:22:07.433873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429709+0000","last_change":"2024-04-02T16:22:01.407141+0000","last_active":"2024-04-02T16:22:07.429709+0000","last_peered":"2024-04-02T16:22:07.429709+0000","last_clean":"2024-04-02T16:22:07.429709+0000","last_became_active":"2024-04-02T16:22:01.403018+0000","last_became_peered":"2024-04-02T16:22:01.403018+0000","last_unstale":"2024-04-02T16:22:07.429709+0000","last_undegraded":"2024-04-02T16:22:07.429709+0000","last_fullsized":"2024-04-02T16:22:07.429709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435416+0000","last_change":"2024-04-02T16:22:01.403712+0000","last_active":"2024-04-02T16:22:07.435416+0000","last_peered":"2024-04-02T16:22:07.435416+0000","last_clean":"2024-04-02T16:22:07.435416+0000","last_became_active":"2024-04-02T16:22:01.402419+0000","last_became_peered":"2024-04-02T16:22:01.402419+0000","last_unstale":"2024-04-02T16:22:07.435416+0000","last_undegraded":"2024-04-02T16:22:07.435416+0000","last_fullsized":"2024-04-02T16:22:07.435416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428475+0000","last_change":"2024-04-02T16:22:01.411196+0000","last_active":"2024-04-02T16:22:07.428475+0000","last_peered":"2024-04-02T16:22:07.428475+0000","last_clean":"2024-04-02T16:22:07.428475+0000","last_became_active":"2024-04-02T16:22:01.410630+0000","last_became_peered":"2024-04-02T16:22:01.410630+0000","last_unstale":"2024-04-02T16:22:07.428475+0000","last_undegraded":"2024-04-02T16:22:07.428475+0000","last_fullsized":"2024-04-02T16:22:07.428475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434802+0000","last_change":"2024-04-02T16:22:01.401113+0000","last_active":"2024-04-02T16:22:07.434802+0000","last_peered":"2024-04-02T16:22:07.434802+0000","last_clean":"2024-04-02T16:22:07.434802+0000","last_became_active":"2024-04-02T16:22:01.400823+0000","last_became_peered":"2024-04-02T16:22:01.400823+0000","last_unstale":"2024-04-02T16:22:07.434802+0000","last_undegraded":"2024-04-02T16:22:07.434802+0000","last_fullsized":"2024-04-02T16:22:07.434802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435248+0000","last_change":"2024-04-02T16:22:01.401273+0000","last_active":"2024-04-02T16:22:07.435248+0000","last_peered":"2024-04-02T16:22:07.435248+0000","last_clean":"2024-04-02T16:22:07.435248+0000","last_became_active":"2024-04-02T16:22:01.400989+0000","last_became_peered":"2024-04-02T16:22:01.400989+0000","last_unstale":"2024-04-02T16:22:07.435248+0000","last_undegraded":"2024-04-02T16:22:07.435248+0000","last_fullsized":"2024-04-02T16:22:07.435248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428640+0000","last_change":"2024-04-02T16:22:01.410167+0000","last_active":"2024-04-02T16:22:07.428640+0000","last_peered":"2024-04-02T16:22:07.428640+0000","last_clean":"2024-04-02T16:22:07.428640+0000","last_became_active":"2024-04-02T16:22:01.409747+0000","last_became_peered":"2024-04-02T16:22:01.409747+0000","last_unstale":"2024-04-02T16:22:07.428640+0000","last_undegraded":"2024-04-02T16:22:07.428640+0000","last_fullsized":"2024-04-02T16:22:07.428640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430485+0000","last_change":"2024-04-02T16:22:01.404722+0000","last_active":"2024-04-02T16:22:07.430485+0000","last_peered":"2024-04-02T16:22:07.430485+0000","last_clean":"2024-04-02T16:22:07.430485+0000","last_became_active":"2024-04-02T16:22:01.403645+0000","last_became_peered":"2024-04-02T16:22:01.403645+0000","last_unstale":"2024-04-02T16:22:07.430485+0000","last_undegraded":"2024-04-02T16:22:07.430485+0000","last_fullsized":"2024-04-02T16:22:07.430485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427877+0000","last_change":"2024-04-02T16:22:01.410096+0000","last_active":"2024-04-02T16:22:07.427877+0000","last_peered":"2024-04-02T16:22:07.427877+0000","last_clean":"2024-04-02T16:22:07.427877+0000","last_became_active":"2024-04-02T16:22:01.409805+0000","last_became_peered":"2024-04-02T16:22:01.409805+0000","last_unstale":"2024-04-02T16:22:07.427877+0000","last_undegraded":"2024-04-02T16:22:07.427877+0000","last_fullsized":"2024-04-02T16:22:07.427877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429582+0000","last_change":"2024-04-02T16:22:01.410138+0000","last_active":"2024-04-02T16:22:07.429582+0000","last_peered":"2024-04-02T16:22:07.429582+0000","last_clean":"2024-04-02T16:22:07.429582+0000","last_became_active":"2024-04-02T16:22:01.403179+0000","last_became_peered":"2024-04-02T16:22:01.403179+0000","last_unstale":"2024-04-02T16:22:07.429582+0000","last_undegraded":"2024-04-02T16:22:07.429582+0000","last_fullsized":"2024-04-02T16:22:07.429582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435300+0000","last_change":"2024-04-02T16:22:01.403457+0000","last_active":"2024-04-02T16:22:07.435300+0000","last_peered":"2024-04-02T16:22:07.435300+0000","last_clean":"2024-04-02T16:22:07.435300+0000","last_became_active":"2024-04-02T16:22:01.402335+0000","last_became_peered":"2024-04-02T16:22:01.402335+0000","last_unstale":"2024-04-02T16:22:07.435300+0000","last_undegraded":"2024-04-02T16:22:07.435300+0000","last_fullsized":"2024-04-02T16:22:07.435300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434948+0000","last_change":"2024-04-02T16:22:01.406544+0000","last_active":"2024-04-02T16:22:07.434948+0000","last_peered":"2024-04-02T16:22:07.434948+0000","last_clean":"2024-04-02T16:22:07.434948+0000","last_became_active":"2024-04-02T16:22:01.406205+0000","last_became_peered":"2024-04-02T16:22:01.406205+0000","last_unstale":"2024-04-02T16:22:07.434948+0000","last_undegraded":"2024-04-02T16:22:07.434948+0000","last_fullsized":"2024-04-02T16:22:07.434948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435085+0000","last_change":"2024-04-02T16:22:01.401972+0000","last_active":"2024-04-02T16:22:07.435085+0000","last_peered":"2024-04-02T16:22:07.435085+0000","last_clean":"2024-04-02T16:22:07.435085+0000","last_became_active":"2024-04-02T16:22:01.401721+0000","last_became_peered":"2024-04-02T16:22:01.401721+0000","last_unstale":"2024-04-02T16:22:07.435085+0000","last_undegraded":"2024-04-02T16:22:07.435085+0000","last_fullsized":"2024-04-02T16:22:07.435085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:05.479052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433925+0000","last_change":"2024-04-02T16:22:01.406917+0000","last_active":"2024-04-02T16:22:07.433925+0000","last_peered":"2024-04-02T16:22:07.433925+0000","last_clean":"2024-04-02T16:22:07.433925+0000","last_became_active":"2024-04-02T16:22:01.406622+0000","last_became_peered":"2024-04-02T16:22:01.406622+0000","last_unstale":"2024-04-02T16:22:07.433925+0000","last_undegraded":"2024-04-02T16:22:07.433925+0000","last_fullsized":"2024-04-02T16:22:07.433925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430970+0000","last_change":"2024-04-02T16:22:01.403891+0000","last_active":"2024-04-02T16:22:07.430970+0000","last_peered":"2024-04-02T16:22:07.430970+0000","last_clean":"2024-04-02T16:22:07.430970+0000","last_became_active":"2024-04-02T16:22:01.403250+0000","last_became_peered":"2024-04-02T16:22:01.403250+0000","last_unstale":"2024-04-02T16:22:07.430970+0000","last_undegraded":"2024-04-02T16:22:07.430970+0000","last_fullsized":"2024-04-02T16:22:07.430970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430459+0000","last_change":"2024-04-02T16:22:01.410686+0000","last_active":"2024-04-02T16:22:07.430459+0000","last_peered":"2024-04-02T16:22:07.430459+0000","last_clean":"2024-04-02T16:22:07.430459+0000","last_became_active":"2024-04-02T16:22:01.410398+0000","last_became_peered":"2024-04-02T16:22:01.410398+0000","last_unstale":"2024-04-02T16:22:07.430459+0000","last_undegraded":"2024-04-02T16:22:07.430459+0000","last_fullsized":"2024-04-02T16:22:07.430459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.406912+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.406626+0000","last_became_peered":"2024-04-02T16:22:01.406626+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434095+0000","last_change":"2024-04-02T16:22:01.397661+0000","last_active":"2024-04-02T16:22:07.434095+0000","last_peered":"2024-04-02T16:22:07.434095+0000","last_clean":"2024-04-02T16:22:07.434095+0000","last_became_active":"2024-04-02T16:22:01.397335+0000","last_became_peered":"2024-04-02T16:22:01.397335+0000","last_unstale":"2024-04-02T16:22:07.434095+0000","last_undegraded":"2024-04-02T16:22:07.434095+0000","last_fullsized":"2024-04-02T16:22:07.434095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428147+0000","last_change":"2024-04-02T16:22:01.409121+0000","last_active":"2024-04-02T16:22:07.428147+0000","last_peered":"2024-04-02T16:22:07.428147+0000","last_clean":"2024-04-02T16:22:07.428147+0000","last_became_active":"2024-04-02T16:22:01.408765+0000","last_became_peered":"2024-04-02T16:22:01.408765+0000","last_unstale":"2024-04-02T16:22:07.428147+0000","last_undegraded":"2024-04-02T16:22:07.428147+0000","last_fullsized":"2024-04-02T16:22:07.428147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429788+0000","last_change":"2024-04-02T16:22:01.407108+0000","last_active":"2024-04-02T16:22:07.429788+0000","last_peered":"2024-04-02T16:22:07.429788+0000","last_clean":"2024-04-02T16:22:07.429788+0000","last_became_active":"2024-04-02T16:22:01.406836+0000","last_became_peered":"2024-04-02T16:22:01.406836+0000","last_unstale":"2024-04-02T16:22:07.429788+0000","last_undegraded":"2024-04-02T16:22:07.429788+0000","last_fullsized":"2024-04-02T16:22:07.429788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434905+0000","last_change":"2024-04-02T16:22:01.414174+0000","last_active":"2024-04-02T16:22:07.434905+0000","last_peered":"2024-04-02T16:22:07.434905+0000","last_clean":"2024-04-02T16:22:07.434905+0000","last_became_active":"2024-04-02T16:22:01.413903+0000","last_became_peered":"2024-04-02T16:22:01.413903+0000","last_unstale":"2024-04-02T16:22:07.434905+0000","last_undegraded":"2024-04-02T16:22:07.434905+0000","last_fullsized":"2024-04-02T16:22:07.434905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435234+0000","last_change":"2024-04-02T16:22:01.413533+0000","last_active":"2024-04-02T16:22:07.435234+0000","last_peered":"2024-04-02T16:22:07.435234+0000","last_clean":"2024-04-02T16:22:07.435234+0000","last_became_active":"2024-04-02T16:22:01.413266+0000","last_became_peered":"2024-04-02T16:22:01.413266+0000","last_unstale":"2024-04-02T16:22:07.435234+0000","last_undegraded":"2024-04-02T16:22:07.435234+0000","last_fullsized":"2024-04-02T16:22:07.435234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435468+0000","last_change":"2024-04-02T16:22:01.412176+0000","last_active":"2024-04-02T16:22:07.435468+0000","last_peered":"2024-04-02T16:22:07.435468+0000","last_clean":"2024-04-02T16:22:07.435468+0000","last_became_active":"2024-04-02T16:22:01.411793+0000","last_became_peered":"2024-04-02T16:22:01.411793+0000","last_unstale":"2024-04-02T16:22:07.435468+0000","last_undegraded":"2024-04-02T16:22:07.435468+0000","last_fullsized":"2024-04-02T16:22:07.435468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:12.015838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427771+0000","last_change":"2024-04-02T16:22:01.410605+0000","last_active":"2024-04-02T16:22:07.427771+0000","last_peered":"2024-04-02T16:22:07.427771+0000","last_clean":"2024-04-02T16:22:07.427771+0000","last_became_active":"2024-04-02T16:22:01.410237+0000","last_became_peered":"2024-04-02T16:22:01.410237+0000","last_unstale":"2024-04-02T16:22:07.427771+0000","last_undegraded":"2024-04-02T16:22:07.427771+0000","last_fullsized":"2024-04-02T16:22:07.427771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434564+0000","last_change":"2024-04-02T16:22:01.404165+0000","last_active":"2024-04-02T16:22:07.434564+0000","last_peered":"2024-04-02T16:22:07.434564+0000","last_clean":"2024-04-02T16:22:07.434564+0000","last_became_active":"2024-04-02T16:22:01.403907+0000","last_became_peered":"2024-04-02T16:22:01.403907+0000","last_unstale":"2024-04-02T16:22:07.434564+0000","last_undegraded":"2024-04-02T16:22:07.434564+0000","last_fullsized":"2024-04-02T16:22:07.434564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429480+0000","last_change":"2024-04-02T16:22:01.409810+0000","last_active":"2024-04-02T16:22:07.429480+0000","last_peered":"2024-04-02T16:22:07.429480+0000","last_clean":"2024-04-02T16:22:07.429480+0000","last_became_active":"2024-04-02T16:22:01.406447+0000","last_became_peered":"2024-04-02T16:22:01.406447+0000","last_unstale":"2024-04-02T16:22:07.429480+0000","last_undegraded":"2024-04-02T16:22:07.429480+0000","last_fullsized":"2024-04-02T16:22:07.429480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434648+0000","last_change":"2024-04-02T16:22:01.401898+0000","last_active":"2024-04-02T16:22:07.434648+0000","last_peered":"2024-04-02T16:22:07.434648+0000","last_clean":"2024-04-02T16:22:07.434648+0000","last_became_active":"2024-04-02T16:22:01.401568+0000","last_became_peered":"2024-04-02T16:22:01.401568+0000","last_unstale":"2024-04-02T16:22:07.434648+0000","last_undegraded":"2024-04-02T16:22:07.434648+0000","last_fullsized":"2024-04-02T16:22:07.434648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430221+0000","last_change":"2024-04-02T16:22:01.410551+0000","last_active":"2024-04-02T16:22:07.430221+0000","last_peered":"2024-04-02T16:22:07.430221+0000","last_clean":"2024-04-02T16:22:07.430221+0000","last_became_active":"2024-04-02T16:22:01.409045+0000","last_became_peered":"2024-04-02T16:22:01.409045+0000","last_unstale":"2024-04-02T16:22:07.430221+0000","last_undegraded":"2024-04-02T16:22:07.430221+0000","last_fullsized":"2024-04-02T16:22:07.430221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434517+0000","last_change":"2024-04-02T16:22:01.398994+0000","last_active":"2024-04-02T16:22:07.434517+0000","last_peered":"2024-04-02T16:22:07.434517+0000","last_clean":"2024-04-02T16:22:07.434517+0000","last_became_active":"2024-04-02T16:22:01.398698+0000","last_became_peered":"2024-04-02T16:22:01.398698+0000","last_unstale":"2024-04-02T16:22:07.434517+0000","last_undegraded":"2024-04-02T16:22:07.434517+0000","last_fullsized":"2024-04-02T16:22:07.434517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435172+0000","last_change":"2024-04-02T16:22:01.406275+0000","last_active":"2024-04-02T16:22:07.435172+0000","last_peered":"2024-04-02T16:22:07.435172+0000","last_clean":"2024-04-02T16:22:07.435172+0000","last_became_active":"2024-04-02T16:22:01.405936+0000","last_became_peered":"2024-04-02T16:22:01.405936+0000","last_unstale":"2024-04-02T16:22:07.435172+0000","last_undegraded":"2024-04-02T16:22:07.435172+0000","last_fullsized":"2024-04-02T16:22:07.435172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435388+0000","last_change":"2024-04-02T16:22:01.407470+0000","last_active":"2024-04-02T16:22:07.435388+0000","last_peered":"2024-04-02T16:22:07.435388+0000","last_clean":"2024-04-02T16:22:07.435388+0000","last_became_active":"2024-04-02T16:22:01.407209+0000","last_became_peered":"2024-04-02T16:22:01.407209+0000","last_unstale":"2024-04-02T16:22:07.435388+0000","last_undegraded":"2024-04-02T16:22:07.435388+0000","last_fullsized":"2024-04-02T16:22:07.435388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:31.303906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:22:24.489 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:22:24.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.622+0000 7f4cd14d3640 1 Processor -- start 2024-04-02T16:22:24.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.622+0000 7f4cd14d3640 1 -- start start 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cd14d3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc15e510 0x7f4ccc0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cd14d3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ccc0d2800 0x7f4ccc0d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cd14d3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ccc0ceda0 con 0x7f4ccc0d0e90 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cd14d3640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f4ccc0d31c0 con 0x7f4ccc0d2800 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cd14d3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ccc0cf040 con 0x7f4ccc15e510 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cca7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ccc0d2800 0x7f4ccc0d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cca7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4ccc0d2800 0x7f4ccc0d2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40088/0 (socket says 172.21.15.50:40088) 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cca7fc640 1 -- 172.21.15.50:0/3609582551 learned_addr learned my addr 172.21.15.50:0/3609582551 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4ccaffd640 1 --2- 172.21.15.50:0/3609582551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc15e510 0x7f4ccc0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cca7fc640 1 -- 172.21.15.50:0/3609582551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc15e510 msgr2=0x7f4ccc0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cca7fc640 1 --2- 172.21.15.50:0/3609582551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc15e510 0x7f4ccc0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cca7fc640 1 -- 172.21.15.50:0/3609582551 >> v1:172.21.15.188:6789/0 conn(0x7f4ccc0d0e90 legacy=0x7f4ccc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.623+0000 7f4cca7fc640 1 -- 172.21.15.50:0/3609582551 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc0cf320 con 0x7f4ccc0d2800 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.624+0000 7f4cca7fc640 1 --2- 172.21.15.50:0/3609582551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc0d2800 0x7f4ccc0d2c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009870 tx=0x7f4cb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc14ba4cb956b37f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.624+0000 7f4cc9ffb640 1 -- 172.21.15.50:0/3609582551 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4035020 con 0x7f4ccc0d2800 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.624+0000 7f4cc9ffb640 1 -- 172.21.15.50:0/3609582551 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb4031640 con 0x7f4ccc0d2800 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.624+0000 7f4cc9ffb640 1 -- 172.21.15.50:0/3609582551 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb402d040 con 0x7f4ccc0d2800 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 -- 172.21.15.50:0/3609582551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc0d2800 msgr2=0x7f4ccc0d2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/3609582551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc0d2800 0x7f4ccc0d2c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009870 tx=0x7f4cb402b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 -- 172.21.15.50:0/3609582551 shutdown_connections 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/3609582551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc0d2800 0x7f4ccc0d2c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/3609582551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc15e510 0x7f4ccc0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 -- 172.21.15.50:0/3609582551 >> 172.21.15.50:0/3609582551 conn(0x7f4ccc09d570 msgr2=0x7f4ccc0d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 -- 172.21.15.50:0/3609582551 shutdown_connections 2024-04-02T16:22:24.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.625+0000 7f4cd14d3640 1 -- 172.21.15.50:0/3609582551 wait complete. 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.626+0000 7f4cd14d3640 1 Processor -- start 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.626+0000 7f4cd14d3640 1 -- start start 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4cd14d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0d0e90 0x7f4ccc1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4cd14d3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc0d2800 0x7f4ccc0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4cd14d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc15e510 0x7f4ccc160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4cd14d3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc0b3ad0 con 0x7f4ccc15e510 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4cd14d3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc15e8f0 con 0x7f4ccc0d0e90 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4cd14d3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ccc0b3c50 con 0x7f4ccc0d2800 2024-04-02T16:22:24.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc15e510 0x7f4ccc160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc15e510 0x7f4ccc160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40102/0 (socket says 172.21.15.50:40102) 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 -- 172.21.15.50:0/747839290 learned_addr learned my addr 172.21.15.50:0/747839290 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccaffd640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0d0e90 0x7f4ccc1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4cca7fc640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc0d2800 0x7f4ccc0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 -- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc0d2800 msgr2=0x7f4ccc0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc0d2800 0x7f4ccc0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 -- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0d0e90 msgr2=0x7f4ccc1f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0d0e90 0x7f4ccc1f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.627+0000 7f4ccb7fe640 1 -- 172.21.15.50:0/747839290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc0cd6f0 con 0x7f4ccc15e510 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.628+0000 7f4ccb7fe640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc15e510 0x7f4ccc160e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4001eb0 tx=0x7f4cc400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.628+0000 7f4cc37fe640 1 -- 172.21.15.50:0/747839290 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4017070 con 0x7f4ccc15e510 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.628+0000 7f4cc37fe640 1 -- 172.21.15.50:0/747839290 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc40052d0 con 0x7f4ccc15e510 2024-04-02T16:22:24.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.628+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ccc0cd9d0 con 0x7f4ccc15e510 2024-04-02T16:22:24.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.628+0000 7f4cc37fe640 1 -- 172.21.15.50:0/747839290 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4003810 con 0x7f4ccc15e510 2024-04-02T16:22:24.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.628+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ccc0cdf90 con 0x7f4ccc15e510 2024-04-02T16:22:24.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.629+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ccc0756f0 con 0x7f4ccc15e510 2024-04-02T16:22:24.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.629+0000 7f4cc37fe640 1 -- 172.21.15.50:0/747839290 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cc401a070 con 0x7f4ccc15e510 2024-04-02T16:22:24.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.629+0000 7f4cc37fe640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4cac06c8f0 0x7f4cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.630+0000 7f4cc37fe640 1 -- 172.21.15.50:0/747839290 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4cc408b140 con 0x7f4ccc15e510 2024-04-02T16:22:24.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.630+0000 7f4ccaffd640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4cac06c8f0 0x7f4cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.630+0000 7f4ccaffd640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4cac06c8f0 0x7f4cac06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc003aa0 tx=0x7f4cbc009830 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.634+0000 7f4cc37fe640 1 -- 172.21.15.50:0/747839290 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4cc4058be0 con 0x7f4ccc15e510 2024-04-02T16:22:24.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.815+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 --> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ccc0b4dc0 con 0x7f4cac06c8f0 2024-04-02T16:22:24.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.824+0000 7f4cc37fe640 1 -- 172.21.15.50:0/747839290 <== mgr.4112 v2:172.21.15.188:6832/2785600878 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f4ccc0b4dc0 con 0x7f4cac06c8f0 2024-04-02T16:22:24.829 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:24.831 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-04-02T16:22:24.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4cac06c8f0 msgr2=0x7f4cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4cac06c8f0 0x7f4cac06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc003aa0 tx=0x7f4cbc009830 comp rx=0 tx=0).stop 2024-04-02T16:22:24.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc15e510 msgr2=0x7f4ccc160e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc15e510 0x7f4ccc160e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4001eb0 tx=0x7f4cc400c890 comp rx=0 tx=0).stop 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 shutdown_connections 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4cac06c8f0 0x7f4cac06edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ccc15e510 0x7f4ccc160e40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc0d2800 0x7f4ccc0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 --2- 172.21.15.50:0/747839290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0d0e90 0x7f4ccc1f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 >> 172.21.15.50:0/747839290 conn(0x7f4ccc09d570 msgr2=0x7f4ccc0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 shutdown_connections 2024-04-02T16:22:24.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.828+0000 7f4cd14d3640 1 -- 172.21.15.50:0/747839290 wait complete. 2024-04-02T16:22:24.842 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-02T16:22:24.671554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236104,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738616,"statfs":{"total":773094113280,"available":772852342784,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002935"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434993+0000","last_change":"2024-04-02T16:22:00.392396+0000","last_active":"2024-04-02T16:22:07.434993+0000","last_peered":"2024-04-02T16:22:07.434993+0000","last_clean":"2024-04-02T16:22:07.434993+0000","last_became_active":"2024-04-02T16:22:00.392089+0000","last_became_peered":"2024-04-02T16:22:00.392089+0000","last_unstale":"2024-04-02T16:22:07.434993+0000","last_undegraded":"2024-04-02T16:22:07.434993+0000","last_fullsized":"2024-04-02T16:22:07.434993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428539+0000","last_change":"2024-04-02T16:22:01.410877+0000","last_active":"2024-04-02T16:22:07.428539+0000","last_peered":"2024-04-02T16:22:07.428539+0000","last_clean":"2024-04-02T16:22:07.428539+0000","last_became_active":"2024-04-02T16:22:01.410574+0000","last_became_peered":"2024-04-02T16:22:01.410574+0000","last_unstale":"2024-04-02T16:22:07.428539+0000","last_undegraded":"2024-04-02T16:22:07.428539+0000","last_fullsized":"2024-04-02T16:22:07.428539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427767+0000","last_change":"2024-04-02T16:22:05.817506+0000","last_active":"2024-04-02T16:22:07.427767+0000","last_peered":"2024-04-02T16:22:07.427767+0000","last_clean":"2024-04-02T16:22:07.427767+0000","last_became_active":"2024-04-02T16:22:05.817145+0000","last_became_peered":"2024-04-02T16:22:05.817145+0000","last_unstale":"2024-04-02T16:22:07.427767+0000","last_undegraded":"2024-04-02T16:22:07.427767+0000","last_fullsized":"2024-04-02T16:22:07.427767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:30.473603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428664+0000","last_change":"2024-04-02T16:22:00.391720+0000","last_active":"2024-04-02T16:22:07.428664+0000","last_peered":"2024-04-02T16:22:07.428664+0000","last_clean":"2024-04-02T16:22:07.428664+0000","last_became_active":"2024-04-02T16:22:00.391262+0000","last_became_peered":"2024-04-02T16:22:00.391262+0000","last_unstale":"2024-04-02T16:22:07.428664+0000","last_undegraded":"2024-04-02T16:22:07.428664+0000","last_fullsized":"2024-04-02T16:22:07.428664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434717+0000","last_change":"2024-04-02T16:22:01.403096+0000","last_active":"2024-04-02T16:22:07.434717+0000","last_peered":"2024-04-02T16:22:07.434717+0000","last_clean":"2024-04-02T16:22:07.434717+0000","last_became_active":"2024-04-02T16:22:01.402805+0000","last_became_peered":"2024-04-02T16:22:01.402805+0000","last_unstale":"2024-04-02T16:22:07.434717+0000","last_undegraded":"2024-04-02T16:22:07.434717+0000","last_fullsized":"2024-04-02T16:22:07.434717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428967+0000","last_change":"2024-04-02T16:22:05.447225+0000","last_active":"2024-04-02T16:22:07.428967+0000","last_peered":"2024-04-02T16:22:07.428967+0000","last_clean":"2024-04-02T16:22:07.428967+0000","last_became_active":"2024-04-02T16:22:05.446843+0000","last_became_peered":"2024-04-02T16:22:05.446843+0000","last_unstale":"2024-04-02T16:22:07.428967+0000","last_undegraded":"2024-04-02T16:22:07.428967+0000","last_fullsized":"2024-04-02T16:22:07.428967+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:00.390875+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:00.390501+0000","last_became_peered":"2024-04-02T16:22:00.390501+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430302+0000","last_change":"2024-04-02T16:22:01.410933+0000","last_active":"2024-04-02T16:22:07.430302+0000","last_peered":"2024-04-02T16:22:07.430302+0000","last_clean":"2024-04-02T16:22:07.430302+0000","last_became_active":"2024-04-02T16:22:01.410242+0000","last_became_peered":"2024-04-02T16:22:01.410242+0000","last_unstale":"2024-04-02T16:22:07.430302+0000","last_undegraded":"2024-04-02T16:22:07.430302+0000","last_fullsized":"2024-04-02T16:22:07.430302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429559+0000","last_change":"2024-04-02T16:22:05.440230+0000","last_active":"2024-04-02T16:22:07.429559+0000","last_peered":"2024-04-02T16:22:07.429559+0000","last_clean":"2024-04-02T16:22:07.429559+0000","last_became_active":"2024-04-02T16:22:05.439599+0000","last_became_peered":"2024-04-02T16:22:05.439599+0000","last_unstale":"2024-04-02T16:22:07.429559+0000","last_undegraded":"2024-04-02T16:22:07.429559+0000","last_fullsized":"2024-04-02T16:22:07.429559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:35.350953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434504+0000","last_change":"2024-04-02T16:22:00.392521+0000","last_active":"2024-04-02T16:22:07.434504+0000","last_peered":"2024-04-02T16:22:07.434504+0000","last_clean":"2024-04-02T16:22:07.434504+0000","last_became_active":"2024-04-02T16:22:00.392244+0000","last_became_peered":"2024-04-02T16:22:00.392244+0000","last_unstale":"2024-04-02T16:22:07.434504+0000","last_undegraded":"2024-04-02T16:22:07.434504+0000","last_fullsized":"2024-04-02T16:22:07.434504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430629+0000","last_change":"2024-04-02T16:22:01.406612+0000","last_active":"2024-04-02T16:22:07.430629+0000","last_peered":"2024-04-02T16:22:07.430629+0000","last_clean":"2024-04-02T16:22:07.430629+0000","last_became_active":"2024-04-02T16:22:01.404789+0000","last_became_peered":"2024-04-02T16:22:01.404789+0000","last_unstale":"2024-04-02T16:22:07.430629+0000","last_undegraded":"2024-04-02T16:22:07.430629+0000","last_fullsized":"2024-04-02T16:22:07.430629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429046+0000","last_change":"2024-04-02T16:22:05.445479+0000","last_active":"2024-04-02T16:22:07.429046+0000","last_peered":"2024-04-02T16:22:07.429046+0000","last_clean":"2024-04-02T16:22:07.429046+0000","last_became_active":"2024-04-02T16:22:05.445022+0000","last_became_peered":"2024-04-02T16:22:05.445022+0000","last_unstale":"2024-04-02T16:22:07.429046+0000","last_undegraded":"2024-04-02T16:22:07.429046+0000","last_fullsized":"2024-04-02T16:22:07.429046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.404940+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.404670+0000","last_became_peered":"2024-04-02T16:22:01.404670+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434832+0000","last_change":"2024-04-02T16:22:00.390811+0000","last_active":"2024-04-02T16:22:07.434832+0000","last_peered":"2024-04-02T16:22:07.434832+0000","last_clean":"2024-04-02T16:22:07.434832+0000","last_became_active":"2024-04-02T16:22:00.390515+0000","last_became_peered":"2024-04-02T16:22:00.390515+0000","last_unstale":"2024-04-02T16:22:07.434832+0000","last_undegraded":"2024-04-02T16:22:07.434832+0000","last_fullsized":"2024-04-02T16:22:07.434832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429947+0000","last_change":"2024-04-02T16:22:05.445431+0000","last_active":"2024-04-02T16:22:07.429947+0000","last_peered":"2024-04-02T16:22:07.429947+0000","last_clean":"2024-04-02T16:22:07.429947+0000","last_became_active":"2024-04-02T16:22:05.445014+0000","last_became_peered":"2024-04-02T16:22:05.445014+0000","last_unstale":"2024-04-02T16:22:07.429947+0000","last_undegraded":"2024-04-02T16:22:07.429947+0000","last_fullsized":"2024-04-02T16:22:07.429947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428171+0000","last_change":"2024-04-02T16:22:01.409289+0000","last_active":"2024-04-02T16:22:07.428171+0000","last_peered":"2024-04-02T16:22:07.428171+0000","last_clean":"2024-04-02T16:22:07.428171+0000","last_became_active":"2024-04-02T16:22:01.408934+0000","last_became_peered":"2024-04-02T16:22:01.408934+0000","last_unstale":"2024-04-02T16:22:07.428171+0000","last_undegraded":"2024-04-02T16:22:07.428171+0000","last_fullsized":"2024-04-02T16:22:07.428171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434963+0000","last_change":"2024-04-02T16:22:00.390727+0000","last_active":"2024-04-02T16:22:07.434963+0000","last_peered":"2024-04-02T16:22:07.434963+0000","last_clean":"2024-04-02T16:22:07.434963+0000","last_became_active":"2024-04-02T16:22:00.390131+0000","last_became_peered":"2024-04-02T16:22:00.390131+0000","last_unstale":"2024-04-02T16:22:07.434963+0000","last_undegraded":"2024-04-02T16:22:07.434963+0000","last_fullsized":"2024-04-02T16:22:07.434963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429330+0000","last_change":"2024-04-02T16:22:05.447195+0000","last_active":"2024-04-02T16:22:07.429330+0000","last_peered":"2024-04-02T16:22:07.429330+0000","last_clean":"2024-04-02T16:22:07.429330+0000","last_became_active":"2024-04-02T16:22:05.446753+0000","last_became_peered":"2024-04-02T16:22:05.446753+0000","last_unstale":"2024-04-02T16:22:07.429330+0000","last_undegraded":"2024-04-02T16:22:07.429330+0000","last_fullsized":"2024-04-02T16:22:07.429330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427951+0000","last_change":"2024-04-02T16:22:01.400634+0000","last_active":"2024-04-02T16:22:07.427951+0000","last_peered":"2024-04-02T16:22:07.427951+0000","last_clean":"2024-04-02T16:22:07.427951+0000","last_became_active":"2024-04-02T16:22:01.400309+0000","last_became_peered":"2024-04-02T16:22:01.400309+0000","last_unstale":"2024-04-02T16:22:07.427951+0000","last_undegraded":"2024-04-02T16:22:07.427951+0000","last_fullsized":"2024-04-02T16:22:07.427951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434579+0000","last_change":"2024-04-02T16:22:00.391494+0000","last_active":"2024-04-02T16:22:07.434579+0000","last_peered":"2024-04-02T16:22:07.434579+0000","last_clean":"2024-04-02T16:22:07.434579+0000","last_became_active":"2024-04-02T16:22:00.391190+0000","last_became_peered":"2024-04-02T16:22:00.391190+0000","last_unstale":"2024-04-02T16:22:07.434579+0000","last_undegraded":"2024-04-02T16:22:07.434579+0000","last_fullsized":"2024-04-02T16:22:07.434579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435264+0000","last_change":"2024-04-02T16:22:05.811534+0000","last_active":"2024-04-02T16:22:07.435264+0000","last_peered":"2024-04-02T16:22:07.435264+0000","last_clean":"2024-04-02T16:22:07.435264+0000","last_became_active":"2024-04-02T16:22:05.811134+0000","last_became_peered":"2024-04-02T16:22:05.811134+0000","last_unstale":"2024-04-02T16:22:07.435264+0000","last_undegraded":"2024-04-02T16:22:07.435264+0000","last_fullsized":"2024-04-02T16:22:07.435264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430330+0000","last_change":"2024-04-02T16:22:01.399287+0000","last_active":"2024-04-02T16:22:07.430330+0000","last_peered":"2024-04-02T16:22:07.430330+0000","last_clean":"2024-04-02T16:22:07.430330+0000","last_became_active":"2024-04-02T16:22:01.398956+0000","last_became_peered":"2024-04-02T16:22:01.398956+0000","last_unstale":"2024-04-02T16:22:07.430330+0000","last_undegraded":"2024-04-02T16:22:07.430330+0000","last_fullsized":"2024-04-02T16:22:07.430330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:50.927884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434912+0000","last_change":"2024-04-02T16:22:00.391289+0000","last_active":"2024-04-02T16:22:07.434912+0000","last_peered":"2024-04-02T16:22:07.434912+0000","last_clean":"2024-04-02T16:22:07.434912+0000","last_became_active":"2024-04-02T16:22:00.390292+0000","last_became_peered":"2024-04-02T16:22:00.390292+0000","last_unstale":"2024-04-02T16:22:07.434912+0000","last_undegraded":"2024-04-02T16:22:07.434912+0000","last_fullsized":"2024-04-02T16:22:07.434912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429938+0000","last_change":"2024-04-02T16:22:05.440165+0000","last_active":"2024-04-02T16:22:07.429938+0000","last_peered":"2024-04-02T16:22:07.429938+0000","last_clean":"2024-04-02T16:22:07.429938+0000","last_became_active":"2024-04-02T16:22:05.439732+0000","last_became_peered":"2024-04-02T16:22:05.439732+0000","last_unstale":"2024-04-02T16:22:07.429938+0000","last_undegraded":"2024-04-02T16:22:07.429938+0000","last_fullsized":"2024-04-02T16:22:07.429938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434684+0000","last_change":"2024-04-02T16:22:01.404915+0000","last_active":"2024-04-02T16:22:07.434684+0000","last_peered":"2024-04-02T16:22:07.434684+0000","last_clean":"2024-04-02T16:22:07.434684+0000","last_became_active":"2024-04-02T16:22:01.402769+0000","last_became_peered":"2024-04-02T16:22:01.402769+0000","last_unstale":"2024-04-02T16:22:07.434684+0000","last_undegraded":"2024-04-02T16:22:07.434684+0000","last_fullsized":"2024-04-02T16:22:07.434684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:45.810039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435410+0000","last_change":"2024-04-02T16:22:00.393479+0000","last_active":"2024-04-02T16:22:07.435410+0000","last_peered":"2024-04-02T16:22:07.435410+0000","last_clean":"2024-04-02T16:22:07.435410+0000","last_became_active":"2024-04-02T16:22:00.393178+0000","last_became_peered":"2024-04-02T16:22:00.393178+0000","last_unstale":"2024-04-02T16:22:07.435410+0000","last_undegraded":"2024-04-02T16:22:07.435410+0000","last_fullsized":"2024-04-02T16:22:07.435410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430212+0000","last_change":"2024-04-02T16:22:05.815460+0000","last_active":"2024-04-02T16:22:07.430212+0000","last_peered":"2024-04-02T16:22:07.430212+0000","last_clean":"2024-04-02T16:22:07.430212+0000","last_became_active":"2024-04-02T16:22:05.814988+0000","last_became_peered":"2024-04-02T16:22:05.814988+0000","last_unstale":"2024-04-02T16:22:07.430212+0000","last_undegraded":"2024-04-02T16:22:07.430212+0000","last_fullsized":"2024-04-02T16:22:07.430212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:01.404044+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:01.403758+0000","last_became_peered":"2024-04-02T16:22:01.403758+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435274+0000","last_change":"2024-04-02T16:22:00.394933+0000","last_active":"2024-04-02T16:22:07.435274+0000","last_peered":"2024-04-02T16:22:07.435274+0000","last_clean":"2024-04-02T16:22:07.435274+0000","last_became_active":"2024-04-02T16:22:00.394672+0000","last_became_peered":"2024-04-02T16:22:00.394672+0000","last_unstale":"2024-04-02T16:22:07.435274+0000","last_undegraded":"2024-04-02T16:22:07.435274+0000","last_fullsized":"2024-04-02T16:22:07.435274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427719+0000","last_change":"2024-04-02T16:22:05.816404+0000","last_active":"2024-04-02T16:22:07.427719+0000","last_peered":"2024-04-02T16:22:07.427719+0000","last_clean":"2024-04-02T16:22:07.427719+0000","last_became_active":"2024-04-02T16:22:05.816011+0000","last_became_peered":"2024-04-02T16:22:05.816011+0000","last_unstale":"2024-04-02T16:22:07.427719+0000","last_undegraded":"2024-04-02T16:22:07.427719+0000","last_fullsized":"2024-04-02T16:22:07.427719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:58.796179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:01.405660+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:01.405343+0000","last_became_peered":"2024-04-02T16:22:01.405343+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435169+0000","last_change":"2024-04-02T16:22:00.391246+0000","last_active":"2024-04-02T16:22:07.435169+0000","last_peered":"2024-04-02T16:22:07.435169+0000","last_clean":"2024-04-02T16:22:07.435169+0000","last_became_active":"2024-04-02T16:22:00.390836+0000","last_became_peered":"2024-04-02T16:22:00.390836+0000","last_unstale":"2024-04-02T16:22:07.435169+0000","last_undegraded":"2024-04-02T16:22:07.435169+0000","last_fullsized":"2024-04-02T16:22:07.435169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:22.952130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435359+0000","last_change":"2024-04-02T16:22:05.815150+0000","last_active":"2024-04-02T16:22:07.435359+0000","last_peered":"2024-04-02T16:22:07.435359+0000","last_clean":"2024-04-02T16:22:07.435359+0000","last_became_active":"2024-04-02T16:22:05.814788+0000","last_became_peered":"2024-04-02T16:22:05.814788+0000","last_unstale":"2024-04-02T16:22:07.435359+0000","last_undegraded":"2024-04-02T16:22:07.435359+0000","last_fullsized":"2024-04-02T16:22:07.435359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430743+0000","last_change":"2024-04-02T16:22:01.404482+0000","last_active":"2024-04-02T16:22:07.430743+0000","last_peered":"2024-04-02T16:22:07.430743+0000","last_clean":"2024-04-02T16:22:07.430743+0000","last_became_active":"2024-04-02T16:22:01.402612+0000","last_became_peered":"2024-04-02T16:22:01.402612+0000","last_unstale":"2024-04-02T16:22:07.430743+0000","last_undegraded":"2024-04-02T16:22:07.430743+0000","last_fullsized":"2024-04-02T16:22:07.430743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435056+0000","last_change":"2024-04-02T16:22:00.388850+0000","last_active":"2024-04-02T16:22:07.435056+0000","last_peered":"2024-04-02T16:22:07.435056+0000","last_clean":"2024-04-02T16:22:07.435056+0000","last_became_active":"2024-04-02T16:22:00.388569+0000","last_became_peered":"2024-04-02T16:22:00.388569+0000","last_unstale":"2024-04-02T16:22:07.435056+0000","last_undegraded":"2024-04-02T16:22:07.435056+0000","last_fullsized":"2024-04-02T16:22:07.435056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434575+0000","last_change":"2024-04-02T16:22:05.445952+0000","last_active":"2024-04-02T16:22:07.434575+0000","last_peered":"2024-04-02T16:22:07.434575+0000","last_clean":"2024-04-02T16:22:07.434575+0000","last_became_active":"2024-04-02T16:22:05.445442+0000","last_became_peered":"2024-04-02T16:22:05.445442+0000","last_unstale":"2024-04-02T16:22:07.434575+0000","last_undegraded":"2024-04-02T16:22:07.434575+0000","last_fullsized":"2024-04-02T16:22:07.434575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429867+0000","last_change":"2024-04-02T16:22:01.405033+0000","last_active":"2024-04-02T16:22:07.429867+0000","last_peered":"2024-04-02T16:22:07.429867+0000","last_clean":"2024-04-02T16:22:07.429867+0000","last_became_active":"2024-04-02T16:22:01.404647+0000","last_became_peered":"2024-04-02T16:22:01.404647+0000","last_unstale":"2024-04-02T16:22:07.429867+0000","last_undegraded":"2024-04-02T16:22:07.429867+0000","last_fullsized":"2024-04-02T16:22:07.429867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:03.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429937+0000","last_change":"2024-04-02T16:22:00.390557+0000","last_active":"2024-04-02T16:22:07.429937+0000","last_peered":"2024-04-02T16:22:07.429937+0000","last_clean":"2024-04-02T16:22:07.429937+0000","last_became_active":"2024-04-02T16:22:00.390250+0000","last_became_peered":"2024-04-02T16:22:00.390250+0000","last_unstale":"2024-04-02T16:22:07.429937+0000","last_undegraded":"2024-04-02T16:22:07.429937+0000","last_fullsized":"2024-04-02T16:22:07.429937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434997+0000","last_change":"2024-04-02T16:22:05.448031+0000","last_active":"2024-04-02T16:22:07.434997+0000","last_peered":"2024-04-02T16:22:07.434997+0000","last_clean":"2024-04-02T16:22:07.434997+0000","last_became_active":"2024-04-02T16:22:05.447648+0000","last_became_peered":"2024-04-02T16:22:05.447648+0000","last_unstale":"2024-04-02T16:22:07.434997+0000","last_undegraded":"2024-04-02T16:22:07.434997+0000","last_fullsized":"2024-04-02T16:22:07.434997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433811+0000","last_change":"2024-04-02T16:22:01.406024+0000","last_active":"2024-04-02T16:22:07.433811+0000","last_peered":"2024-04-02T16:22:07.433811+0000","last_clean":"2024-04-02T16:22:07.433811+0000","last_became_active":"2024-04-02T16:22:01.405628+0000","last_became_peered":"2024-04-02T16:22:01.405628+0000","last_unstale":"2024-04-02T16:22:07.433811+0000","last_undegraded":"2024-04-02T16:22:07.433811+0000","last_fullsized":"2024-04-02T16:22:07.433811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433971+0000","last_change":"2024-04-02T16:22:00.384113+0000","last_active":"2024-04-02T16:22:07.433971+0000","last_peered":"2024-04-02T16:22:07.433971+0000","last_clean":"2024-04-02T16:22:07.433971+0000","last_became_active":"2024-04-02T16:22:00.383827+0000","last_became_peered":"2024-04-02T16:22:00.383827+0000","last_unstale":"2024-04-02T16:22:07.433971+0000","last_undegraded":"2024-04-02T16:22:07.433971+0000","last_fullsized":"2024-04-02T16:22:07.433971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435258+0000","last_change":"2024-04-02T16:22:05.448007+0000","last_active":"2024-04-02T16:22:07.435258+0000","last_peered":"2024-04-02T16:22:07.435258+0000","last_clean":"2024-04-02T16:22:07.435258+0000","last_became_active":"2024-04-02T16:22:05.447605+0000","last_became_peered":"2024-04-02T16:22:05.447605+0000","last_unstale":"2024-04-02T16:22:07.435258+0000","last_undegraded":"2024-04-02T16:22:07.435258+0000","last_fullsized":"2024-04-02T16:22:07.435258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434278+0000","last_change":"2024-04-02T16:22:01.394732+0000","last_active":"2024-04-02T16:22:07.434278+0000","last_peered":"2024-04-02T16:22:07.434278+0000","last_clean":"2024-04-02T16:22:07.434278+0000","last_became_active":"2024-04-02T16:22:01.394437+0000","last_became_peered":"2024-04-02T16:22:01.394437+0000","last_unstale":"2024-04-02T16:22:07.434278+0000","last_undegraded":"2024-04-02T16:22:07.434278+0000","last_fullsized":"2024-04-02T16:22:07.434278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:26:16.963906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434233+0000","last_change":"2024-04-02T16:22:00.391466+0000","last_active":"2024-04-02T16:22:07.434233+0000","last_peered":"2024-04-02T16:22:07.434233+0000","last_clean":"2024-04-02T16:22:07.434233+0000","last_became_active":"2024-04-02T16:22:00.391148+0000","last_became_peered":"2024-04-02T16:22:00.391148+0000","last_unstale":"2024-04-02T16:22:07.434233+0000","last_undegraded":"2024-04-02T16:22:07.434233+0000","last_fullsized":"2024-04-02T16:22:07.434233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434509+0000","last_change":"2024-04-02T16:22:05.811459+0000","last_active":"2024-04-02T16:22:07.434509+0000","last_peered":"2024-04-02T16:22:07.434509+0000","last_clean":"2024-04-02T16:22:07.434509+0000","last_became_active":"2024-04-02T16:22:05.811036+0000","last_became_peered":"2024-04-02T16:22:05.811036+0000","last_unstale":"2024-04-02T16:22:07.434509+0000","last_undegraded":"2024-04-02T16:22:07.434509+0000","last_fullsized":"2024-04-02T16:22:07.434509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428029+0000","last_change":"2024-04-02T16:22:01.410992+0000","last_active":"2024-04-02T16:22:07.428029+0000","last_peered":"2024-04-02T16:22:07.428029+0000","last_clean":"2024-04-02T16:22:07.428029+0000","last_became_active":"2024-04-02T16:22:01.410701+0000","last_became_peered":"2024-04-02T16:22:01.410701+0000","last_unstale":"2024-04-02T16:22:07.428029+0000","last_undegraded":"2024-04-02T16:22:07.428029+0000","last_fullsized":"2024-04-02T16:22:07.428029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429896+0000","last_change":"2024-04-02T16:22:00.382563+0000","last_active":"2024-04-02T16:22:07.429896+0000","last_peered":"2024-04-02T16:22:07.429896+0000","last_clean":"2024-04-02T16:22:07.429896+0000","last_became_active":"2024-04-02T16:22:00.382285+0000","last_became_peered":"2024-04-02T16:22:00.382285+0000","last_unstale":"2024-04-02T16:22:07.429896+0000","last_undegraded":"2024-04-02T16:22:07.429896+0000","last_fullsized":"2024-04-02T16:22:07.429896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:21:49.851609+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:21:49.850564+0000","last_became_peered":"2024-04-02T16:21:49.850564+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_clean_scrub_stamp":"2024-04-02T16:21:48.692321+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.825754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434765+0000","last_change":"2024-04-02T16:22:05.816624+0000","last_active":"2024-04-02T16:22:07.434765+0000","last_peered":"2024-04-02T16:22:07.434765+0000","last_clean":"2024-04-02T16:22:07.434765+0000","last_became_active":"2024-04-02T16:22:05.816230+0000","last_became_peered":"2024-04-02T16:22:05.816230+0000","last_unstale":"2024-04-02T16:22:07.434765+0000","last_undegraded":"2024-04-02T16:22:07.434765+0000","last_fullsized":"2024-04-02T16:22:07.434765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434871+0000","last_change":"2024-04-02T16:22:01.412438+0000","last_active":"2024-04-02T16:22:07.434871+0000","last_peered":"2024-04-02T16:22:07.434871+0000","last_clean":"2024-04-02T16:22:07.434871+0000","last_became_active":"2024-04-02T16:22:01.411845+0000","last_became_peered":"2024-04-02T16:22:01.411845+0000","last_unstale":"2024-04-02T16:22:07.434871+0000","last_undegraded":"2024-04-02T16:22:07.434871+0000","last_fullsized":"2024-04-02T16:22:07.434871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:22:00.381230+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:22:00.380907+0000","last_became_peered":"2024-04-02T16:22:00.380907+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434320+0000","last_change":"2024-04-02T16:22:05.815548+0000","last_active":"2024-04-02T16:22:07.434320+0000","last_peered":"2024-04-02T16:22:07.434320+0000","last_clean":"2024-04-02T16:22:07.434320+0000","last_became_active":"2024-04-02T16:22:05.815073+0000","last_became_peered":"2024-04-02T16:22:05.815073+0000","last_unstale":"2024-04-02T16:22:07.434320+0000","last_undegraded":"2024-04-02T16:22:07.434320+0000","last_fullsized":"2024-04-02T16:22:07.434320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430426+0000","last_change":"2024-04-02T16:22:01.407537+0000","last_active":"2024-04-02T16:22:07.430426+0000","last_peered":"2024-04-02T16:22:07.430426+0000","last_clean":"2024-04-02T16:22:07.430426+0000","last_became_active":"2024-04-02T16:22:01.403892+0000","last_became_peered":"2024-04-02T16:22:01.403892+0000","last_unstale":"2024-04-02T16:22:07.430426+0000","last_undegraded":"2024-04-02T16:22:07.430426+0000","last_fullsized":"2024-04-02T16:22:07.430426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434911+0000","last_change":"2024-04-02T16:22:00.387875+0000","last_active":"2024-04-02T16:22:07.434911+0000","last_peered":"2024-04-02T16:22:07.434911+0000","last_clean":"2024-04-02T16:22:07.434911+0000","last_became_active":"2024-04-02T16:22:00.387562+0000","last_became_peered":"2024-04-02T16:22:00.387562+0000","last_unstale":"2024-04-02T16:22:07.434911+0000","last_undegraded":"2024-04-02T16:22:07.434911+0000","last_fullsized":"2024-04-02T16:22:07.434911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435508+0000","last_change":"2024-04-02T16:22:05.450999+0000","last_active":"2024-04-02T16:22:07.435508+0000","last_peered":"2024-04-02T16:22:07.435508+0000","last_clean":"2024-04-02T16:22:07.435508+0000","last_became_active":"2024-04-02T16:22:05.450407+0000","last_became_peered":"2024-04-02T16:22:05.450407+0000","last_unstale":"2024-04-02T16:22:07.435508+0000","last_undegraded":"2024-04-02T16:22:07.435508+0000","last_fullsized":"2024-04-02T16:22:07.435508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435483+0000","last_change":"2024-04-02T16:22:01.410592+0000","last_active":"2024-04-02T16:22:07.435483+0000","last_peered":"2024-04-02T16:22:07.435483+0000","last_clean":"2024-04-02T16:22:07.435483+0000","last_became_active":"2024-04-02T16:22:01.410288+0000","last_became_peered":"2024-04-02T16:22:01.410288+0000","last_unstale":"2024-04-02T16:22:07.435483+0000","last_undegraded":"2024-04-02T16:22:07.435483+0000","last_fullsized":"2024-04-02T16:22:07.435483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:58.411440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434019+0000","last_change":"2024-04-02T16:22:00.389059+0000","last_active":"2024-04-02T16:22:07.434019+0000","last_peered":"2024-04-02T16:22:07.434019+0000","last_clean":"2024-04-02T16:22:07.434019+0000","last_became_active":"2024-04-02T16:22:00.388780+0000","last_became_peered":"2024-04-02T16:22:00.388780+0000","last_unstale":"2024-04-02T16:22:07.434019+0000","last_undegraded":"2024-04-02T16:22:07.434019+0000","last_fullsized":"2024-04-02T16:22:07.434019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430338+0000","last_change":"2024-04-02T16:22:05.811513+0000","last_active":"2024-04-02T16:22:07.430338+0000","last_peered":"2024-04-02T16:22:07.430338+0000","last_clean":"2024-04-02T16:22:07.430338+0000","last_became_active":"2024-04-02T16:22:05.811011+0000","last_became_peered":"2024-04-02T16:22:05.811011+0000","last_unstale":"2024-04-02T16:22:07.430338+0000","last_undegraded":"2024-04-02T16:22:07.430338+0000","last_fullsized":"2024-04-02T16:22:07.430338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435147+0000","last_change":"2024-04-02T16:22:01.404383+0000","last_active":"2024-04-02T16:22:07.435147+0000","last_peered":"2024-04-02T16:22:07.435147+0000","last_clean":"2024-04-02T16:22:07.435147+0000","last_became_active":"2024-04-02T16:22:01.404105+0000","last_became_peered":"2024-04-02T16:22:01.404105+0000","last_unstale":"2024-04-02T16:22:07.435147+0000","last_undegraded":"2024-04-02T16:22:07.435147+0000","last_fullsized":"2024-04-02T16:22:07.435147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429344+0000","last_change":"2024-04-02T16:22:00.385712+0000","last_active":"2024-04-02T16:22:07.429344+0000","last_peered":"2024-04-02T16:22:07.429344+0000","last_clean":"2024-04-02T16:22:07.429344+0000","last_became_active":"2024-04-02T16:22:00.385374+0000","last_became_peered":"2024-04-02T16:22:00.385374+0000","last_unstale":"2024-04-02T16:22:07.429344+0000","last_undegraded":"2024-04-02T16:22:07.429344+0000","last_fullsized":"2024-04-02T16:22:07.429344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430015+0000","last_change":"2024-04-02T16:22:05.439393+0000","last_active":"2024-04-02T16:22:07.430015+0000","last_peered":"2024-04-02T16:22:07.430015+0000","last_clean":"2024-04-02T16:22:07.430015+0000","last_became_active":"2024-04-02T16:22:05.439022+0000","last_became_peered":"2024-04-02T16:22:05.439022+0000","last_unstale":"2024-04-02T16:22:07.430015+0000","last_undegraded":"2024-04-02T16:22:07.430015+0000","last_fullsized":"2024-04-02T16:22:07.430015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434659+0000","last_change":"2024-04-02T16:22:01.412452+0000","last_active":"2024-04-02T16:22:07.434659+0000","last_peered":"2024-04-02T16:22:07.434659+0000","last_clean":"2024-04-02T16:22:07.434659+0000","last_became_active":"2024-04-02T16:22:01.412160+0000","last_became_peered":"2024-04-02T16:22:01.412160+0000","last_unstale":"2024-04-02T16:22:07.434659+0000","last_undegraded":"2024-04-02T16:22:07.434659+0000","last_fullsized":"2024-04-02T16:22:07.434659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433532+0000","last_change":"2024-04-02T16:22:00.386167+0000","last_active":"2024-04-02T16:22:07.433532+0000","last_peered":"2024-04-02T16:22:07.433532+0000","last_clean":"2024-04-02T16:22:07.433532+0000","last_became_active":"2024-04-02T16:22:00.385887+0000","last_became_peered":"2024-04-02T16:22:00.385887+0000","last_unstale":"2024-04-02T16:22:07.433532+0000","last_undegraded":"2024-04-02T16:22:07.433532+0000","last_fullsized":"2024-04-02T16:22:07.433532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434360+0000","last_change":"2024-04-02T16:22:05.451144+0000","last_active":"2024-04-02T16:22:07.434360+0000","last_peered":"2024-04-02T16:22:07.434360+0000","last_clean":"2024-04-02T16:22:07.434360+0000","last_became_active":"2024-04-02T16:22:05.450690+0000","last_became_peered":"2024-04-02T16:22:05.450690+0000","last_unstale":"2024-04-02T16:22:07.434360+0000","last_undegraded":"2024-04-02T16:22:07.434360+0000","last_fullsized":"2024-04-02T16:22:07.434360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434308+0000","last_change":"2024-04-02T16:22:01.409326+0000","last_active":"2024-04-02T16:22:07.434308+0000","last_peered":"2024-04-02T16:22:07.434308+0000","last_clean":"2024-04-02T16:22:07.434308+0000","last_became_active":"2024-04-02T16:22:01.408710+0000","last_became_peered":"2024-04-02T16:22:01.408710+0000","last_unstale":"2024-04-02T16:22:07.434308+0000","last_undegraded":"2024-04-02T16:22:07.434308+0000","last_fullsized":"2024-04-02T16:22:07.434308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434473+0000","last_change":"2024-04-02T16:22:00.389406+0000","last_active":"2024-04-02T16:22:07.434473+0000","last_peered":"2024-04-02T16:22:07.434473+0000","last_clean":"2024-04-02T16:22:07.434473+0000","last_became_active":"2024-04-02T16:22:00.389107+0000","last_became_peered":"2024-04-02T16:22:00.389107+0000","last_unstale":"2024-04-02T16:22:07.434473+0000","last_undegraded":"2024-04-02T16:22:07.434473+0000","last_fullsized":"2024-04-02T16:22:07.434473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:05.816005+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:45.290851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434745+0000","last_change":"2024-04-02T16:22:01.412447+0000","last_active":"2024-04-02T16:22:07.434745+0000","last_peered":"2024-04-02T16:22:07.434745+0000","last_clean":"2024-04-02T16:22:07.434745+0000","last_became_active":"2024-04-02T16:22:01.411677+0000","last_became_peered":"2024-04-02T16:22:01.411677+0000","last_unstale":"2024-04-02T16:22:07.434745+0000","last_undegraded":"2024-04-02T16:22:07.434745+0000","last_fullsized":"2024-04-02T16:22:07.434745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:00.389887+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:00.389287+0000","last_became_peered":"2024-04-02T16:22:00.389287+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428399+0000","last_change":"2024-04-02T16:22:05.815236+0000","last_active":"2024-04-02T16:22:07.428399+0000","last_peered":"2024-04-02T16:22:07.428399+0000","last_clean":"2024-04-02T16:22:07.428399+0000","last_became_active":"2024-04-02T16:22:05.814782+0000","last_became_peered":"2024-04-02T16:22:05.814782+0000","last_unstale":"2024-04-02T16:22:07.428399+0000","last_undegraded":"2024-04-02T16:22:07.428399+0000","last_fullsized":"2024-04-02T16:22:07.428399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:44.247231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434209+0000","last_change":"2024-04-02T16:22:01.401164+0000","last_active":"2024-04-02T16:22:07.434209+0000","last_peered":"2024-04-02T16:22:07.434209+0000","last_clean":"2024-04-02T16:22:07.434209+0000","last_became_active":"2024-04-02T16:22:01.400880+0000","last_became_peered":"2024-04-02T16:22:01.400880+0000","last_unstale":"2024-04-02T16:22:07.434209+0000","last_undegraded":"2024-04-02T16:22:07.434209+0000","last_fullsized":"2024-04-02T16:22:07.434209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435570+0000","last_change":"2024-04-02T16:22:00.386660+0000","last_active":"2024-04-02T16:22:07.435570+0000","last_peered":"2024-04-02T16:22:07.435570+0000","last_clean":"2024-04-02T16:22:07.435570+0000","last_became_active":"2024-04-02T16:22:00.386370+0000","last_became_peered":"2024-04-02T16:22:00.386370+0000","last_unstale":"2024-04-02T16:22:07.435570+0000","last_undegraded":"2024-04-02T16:22:07.435570+0000","last_fullsized":"2024-04-02T16:22:07.435570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430760+0000","last_change":"2024-04-02T16:22:05.816956+0000","last_active":"2024-04-02T16:22:07.430760+0000","last_peered":"2024-04-02T16:22:07.430760+0000","last_clean":"2024-04-02T16:22:07.430760+0000","last_became_active":"2024-04-02T16:22:05.816429+0000","last_became_peered":"2024-04-02T16:22:05.816429+0000","last_unstale":"2024-04-02T16:22:07.430760+0000","last_undegraded":"2024-04-02T16:22:07.430760+0000","last_fullsized":"2024-04-02T16:22:07.430760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434384+0000","last_change":"2024-04-02T16:22:01.405624+0000","last_active":"2024-04-02T16:22:07.434384+0000","last_peered":"2024-04-02T16:22:07.434384+0000","last_clean":"2024-04-02T16:22:07.434384+0000","last_became_active":"2024-04-02T16:22:01.405347+0000","last_became_peered":"2024-04-02T16:22:01.405347+0000","last_unstale":"2024-04-02T16:22:07.434384+0000","last_undegraded":"2024-04-02T16:22:07.434384+0000","last_fullsized":"2024-04-02T16:22:07.434384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:10.942808+0000","last_change":"2024-04-02T16:22:00.391986+0000","last_active":"2024-04-02T16:22:10.942808+0000","last_peered":"2024-04-02T16:22:10.942808+0000","last_clean":"2024-04-02T16:22:10.942808+0000","last_became_active":"2024-04-02T16:22:00.391681+0000","last_became_peered":"2024-04-02T16:22:00.391681+0000","last_unstale":"2024-04-02T16:22:10.942808+0000","last_undegraded":"2024-04-02T16:22:10.942808+0000","last_fullsized":"2024-04-02T16:22:10.942808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434848+0000","last_change":"2024-04-02T16:22:05.446704+0000","last_active":"2024-04-02T16:22:07.434848+0000","last_peered":"2024-04-02T16:22:07.434848+0000","last_clean":"2024-04-02T16:22:07.434848+0000","last_became_active":"2024-04-02T16:22:05.446150+0000","last_became_peered":"2024-04-02T16:22:05.446150+0000","last_unstale":"2024-04-02T16:22:07.434848+0000","last_undegraded":"2024-04-02T16:22:07.434848+0000","last_fullsized":"2024-04-02T16:22:07.434848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:01.400842+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:01.400565+0000","last_became_peered":"2024-04-02T16:22:01.400565+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434908+0000","last_change":"2024-04-02T16:22:00.394135+0000","last_active":"2024-04-02T16:22:07.434908+0000","last_peered":"2024-04-02T16:22:07.434908+0000","last_clean":"2024-04-02T16:22:07.434908+0000","last_became_active":"2024-04-02T16:22:00.393846+0000","last_became_peered":"2024-04-02T16:22:00.393846+0000","last_unstale":"2024-04-02T16:22:07.434908+0000","last_undegraded":"2024-04-02T16:22:07.434908+0000","last_fullsized":"2024-04-02T16:22:07.434908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429907+0000","last_change":"2024-04-02T16:22:05.439118+0000","last_active":"2024-04-02T16:22:07.429907+0000","last_peered":"2024-04-02T16:22:07.429907+0000","last_clean":"2024-04-02T16:22:07.429907+0000","last_became_active":"2024-04-02T16:22:05.438557+0000","last_became_peered":"2024-04-02T16:22:05.438557+0000","last_unstale":"2024-04-02T16:22:07.429907+0000","last_undegraded":"2024-04-02T16:22:07.429907+0000","last_fullsized":"2024-04-02T16:22:07.429907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434885+0000","last_change":"2024-04-02T16:22:01.413531+0000","last_active":"2024-04-02T16:22:07.434885+0000","last_peered":"2024-04-02T16:22:07.434885+0000","last_clean":"2024-04-02T16:22:07.434885+0000","last_became_active":"2024-04-02T16:22:01.413229+0000","last_became_peered":"2024-04-02T16:22:01.413229+0000","last_unstale":"2024-04-02T16:22:07.434885+0000","last_undegraded":"2024-04-02T16:22:07.434885+0000","last_fullsized":"2024-04-02T16:22:07.434885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430736+0000","last_change":"2024-04-02T16:22:00.388772+0000","last_active":"2024-04-02T16:22:07.430736+0000","last_peered":"2024-04-02T16:22:07.430736+0000","last_clean":"2024-04-02T16:22:07.430736+0000","last_became_active":"2024-04-02T16:22:00.388500+0000","last_became_peered":"2024-04-02T16:22:00.388500+0000","last_unstale":"2024-04-02T16:22:07.430736+0000","last_undegraded":"2024-04-02T16:22:07.430736+0000","last_fullsized":"2024-04-02T16:22:07.430736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427878+0000","last_change":"2024-04-02T16:22:05.441542+0000","last_active":"2024-04-02T16:22:07.427878+0000","last_peered":"2024-04-02T16:22:07.427878+0000","last_clean":"2024-04-02T16:22:07.427878+0000","last_became_active":"2024-04-02T16:22:05.441134+0000","last_became_peered":"2024-04-02T16:22:05.441134+0000","last_unstale":"2024-04-02T16:22:07.427878+0000","last_undegraded":"2024-04-02T16:22:07.427878+0000","last_fullsized":"2024-04-02T16:22:07.427878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:02:44.237629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430701+0000","last_change":"2024-04-02T16:22:01.401315+0000","last_active":"2024-04-02T16:22:07.430701+0000","last_peered":"2024-04-02T16:22:07.430701+0000","last_clean":"2024-04-02T16:22:07.430701+0000","last_became_active":"2024-04-02T16:22:01.401023+0000","last_became_peered":"2024-04-02T16:22:01.401023+0000","last_unstale":"2024-04-02T16:22:07.430701+0000","last_undegraded":"2024-04-02T16:22:07.430701+0000","last_fullsized":"2024-04-02T16:22:07.430701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435375+0000","last_change":"2024-04-02T16:22:00.394082+0000","last_active":"2024-04-02T16:22:07.435375+0000","last_peered":"2024-04-02T16:22:07.435375+0000","last_clean":"2024-04-02T16:22:07.435375+0000","last_became_active":"2024-04-02T16:22:00.393793+0000","last_became_peered":"2024-04-02T16:22:00.393793+0000","last_unstale":"2024-04-02T16:22:07.435375+0000","last_undegraded":"2024-04-02T16:22:07.435375+0000","last_fullsized":"2024-04-02T16:22:07.435375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427851+0000","last_change":"2024-04-02T16:22:05.817432+0000","last_active":"2024-04-02T16:22:07.427851+0000","last_peered":"2024-04-02T16:22:07.427851+0000","last_clean":"2024-04-02T16:22:07.427851+0000","last_became_active":"2024-04-02T16:22:05.817018+0000","last_became_peered":"2024-04-02T16:22:05.817018+0000","last_unstale":"2024-04-02T16:22:07.427851+0000","last_undegraded":"2024-04-02T16:22:07.427851+0000","last_fullsized":"2024-04-02T16:22:07.427851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:52:22.886852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434205+0000","last_change":"2024-04-02T16:22:05.815342+0000","last_active":"2024-04-02T16:22:07.434205+0000","last_peered":"2024-04-02T16:22:07.434205+0000","last_clean":"2024-04-02T16:22:07.434205+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.434205+0000","last_undegraded":"2024-04-02T16:22:07.434205+0000","last_fullsized":"2024-04-02T16:22:07.434205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434398+0000","last_change":"2024-04-02T16:22:01.400394+0000","last_active":"2024-04-02T16:22:07.434398+0000","last_peered":"2024-04-02T16:22:07.434398+0000","last_clean":"2024-04-02T16:22:07.434398+0000","last_became_active":"2024-04-02T16:22:01.400000+0000","last_became_peered":"2024-04-02T16:22:01.400000+0000","last_unstale":"2024-04-02T16:22:07.434398+0000","last_undegraded":"2024-04-02T16:22:07.434398+0000","last_fullsized":"2024-04-02T16:22:07.434398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430602+0000","last_change":"2024-04-02T16:22:00.380328+0000","last_active":"2024-04-02T16:22:07.430602+0000","last_peered":"2024-04-02T16:22:07.430602+0000","last_clean":"2024-04-02T16:22:07.430602+0000","last_became_active":"2024-04-02T16:22:00.379294+0000","last_became_peered":"2024-04-02T16:22:00.379294+0000","last_unstale":"2024-04-02T16:22:07.430602+0000","last_undegraded":"2024-04-02T16:22:07.430602+0000","last_fullsized":"2024-04-02T16:22:07.430602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435187+0000","last_change":"2024-04-02T16:22:05.444408+0000","last_active":"2024-04-02T16:22:07.435187+0000","last_peered":"2024-04-02T16:22:07.435187+0000","last_clean":"2024-04-02T16:22:07.435187+0000","last_became_active":"2024-04-02T16:22:05.444100+0000","last_became_peered":"2024-04-02T16:22:05.444100+0000","last_unstale":"2024-04-02T16:22:07.435187+0000","last_undegraded":"2024-04-02T16:22:07.435187+0000","last_fullsized":"2024-04-02T16:22:07.435187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427872+0000","last_change":"2024-04-02T16:22:01.409456+0000","last_active":"2024-04-02T16:22:07.427872+0000","last_peered":"2024-04-02T16:22:07.427872+0000","last_clean":"2024-04-02T16:22:07.427872+0000","last_became_active":"2024-04-02T16:22:01.408824+0000","last_became_peered":"2024-04-02T16:22:01.408824+0000","last_unstale":"2024-04-02T16:22:07.427872+0000","last_undegraded":"2024-04-02T16:22:07.427872+0000","last_fullsized":"2024-04-02T16:22:07.427872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427970+0000","last_change":"2024-04-02T16:22:00.390487+0000","last_active":"2024-04-02T16:22:07.427970+0000","last_peered":"2024-04-02T16:22:07.427970+0000","last_clean":"2024-04-02T16:22:07.427970+0000","last_became_active":"2024-04-02T16:22:00.390198+0000","last_became_peered":"2024-04-02T16:22:00.390198+0000","last_unstale":"2024-04-02T16:22:07.427970+0000","last_undegraded":"2024-04-02T16:22:07.427970+0000","last_fullsized":"2024-04-02T16:22:07.427970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434683+0000","last_change":"2024-04-02T16:22:00.388624+0000","last_active":"2024-04-02T16:22:07.434683+0000","last_peered":"2024-04-02T16:22:07.434683+0000","last_clean":"2024-04-02T16:22:07.434683+0000","last_became_active":"2024-04-02T16:22:00.388351+0000","last_became_peered":"2024-04-02T16:22:00.388351+0000","last_unstale":"2024-04-02T16:22:07.434683+0000","last_undegraded":"2024-04-02T16:22:07.434683+0000","last_fullsized":"2024-04-02T16:22:07.434683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435511+0000","last_change":"2024-04-02T16:22:01.403982+0000","last_active":"2024-04-02T16:22:07.435511+0000","last_peered":"2024-04-02T16:22:07.435511+0000","last_clean":"2024-04-02T16:22:07.435511+0000","last_became_active":"2024-04-02T16:22:01.403057+0000","last_became_peered":"2024-04-02T16:22:01.403057+0000","last_unstale":"2024-04-02T16:22:07.435511+0000","last_undegraded":"2024-04-02T16:22:07.435511+0000","last_fullsized":"2024-04-02T16:22:07.435511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434167+0000","last_change":"2024-04-02T16:22:05.443373+0000","last_active":"2024-04-02T16:22:07.434167+0000","last_peered":"2024-04-02T16:22:07.434167+0000","last_clean":"2024-04-02T16:22:07.434167+0000","last_became_active":"2024-04-02T16:22:05.442934+0000","last_became_peered":"2024-04-02T16:22:05.442934+0000","last_unstale":"2024-04-02T16:22:07.434167+0000","last_undegraded":"2024-04-02T16:22:07.434167+0000","last_fullsized":"2024-04-02T16:22:07.434167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:49.857086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428264+0000","last_change":"2024-04-02T16:22:00.391680+0000","last_active":"2024-04-02T16:22:07.428264+0000","last_peered":"2024-04-02T16:22:07.428264+0000","last_clean":"2024-04-02T16:22:07.428264+0000","last_became_active":"2024-04-02T16:22:00.391402+0000","last_became_peered":"2024-04-02T16:22:00.391402+0000","last_unstale":"2024-04-02T16:22:07.428264+0000","last_undegraded":"2024-04-02T16:22:07.428264+0000","last_fullsized":"2024-04-02T16:22:07.428264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429224+0000","last_change":"2024-04-02T16:22:01.398824+0000","last_active":"2024-04-02T16:22:07.429224+0000","last_peered":"2024-04-02T16:22:07.429224+0000","last_clean":"2024-04-02T16:22:07.429224+0000","last_became_active":"2024-04-02T16:22:01.398529+0000","last_became_peered":"2024-04-02T16:22:01.398529+0000","last_unstale":"2024-04-02T16:22:07.429224+0000","last_undegraded":"2024-04-02T16:22:07.429224+0000","last_fullsized":"2024-04-02T16:22:07.429224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:01.320427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428686+0000","last_change":"2024-04-02T16:22:05.440423+0000","last_active":"2024-04-02T16:22:07.428686+0000","last_peered":"2024-04-02T16:22:07.428686+0000","last_clean":"2024-04-02T16:22:07.428686+0000","last_became_active":"2024-04-02T16:22:05.439993+0000","last_became_peered":"2024-04-02T16:22:05.439993+0000","last_unstale":"2024-04-02T16:22:07.428686+0000","last_undegraded":"2024-04-02T16:22:07.428686+0000","last_fullsized":"2024-04-02T16:22:07.428686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:17.805386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434177+0000","last_change":"2024-04-02T16:22:01.408902+0000","last_active":"2024-04-02T16:22:07.434177+0000","last_peered":"2024-04-02T16:22:07.434177+0000","last_clean":"2024-04-02T16:22:07.434177+0000","last_became_active":"2024-04-02T16:22:01.408562+0000","last_became_peered":"2024-04-02T16:22:01.408562+0000","last_unstale":"2024-04-02T16:22:07.434177+0000","last_undegraded":"2024-04-02T16:22:07.434177+0000","last_fullsized":"2024-04-02T16:22:07.434177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433873+0000","last_change":"2024-04-02T16:22:01.404001+0000","last_active":"2024-04-02T16:22:07.433873+0000","last_peered":"2024-04-02T16:22:07.433873+0000","last_clean":"2024-04-02T16:22:07.433873+0000","last_became_active":"2024-04-02T16:22:01.403694+0000","last_became_peered":"2024-04-02T16:22:01.403694+0000","last_unstale":"2024-04-02T16:22:07.433873+0000","last_undegraded":"2024-04-02T16:22:07.433873+0000","last_fullsized":"2024-04-02T16:22:07.433873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429709+0000","last_change":"2024-04-02T16:22:01.407141+0000","last_active":"2024-04-02T16:22:07.429709+0000","last_peered":"2024-04-02T16:22:07.429709+0000","last_clean":"2024-04-02T16:22:07.429709+0000","last_became_active":"2024-04-02T16:22:01.403018+0000","last_became_peered":"2024-04-02T16:22:01.403018+0000","last_unstale":"2024-04-02T16:22:07.429709+0000","last_undegraded":"2024-04-02T16:22:07.429709+0000","last_fullsized":"2024-04-02T16:22:07.429709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435416+0000","last_change":"2024-04-02T16:22:01.403712+0000","last_active":"2024-04-02T16:22:07.435416+0000","last_peered":"2024-04-02T16:22:07.435416+0000","last_clean":"2024-04-02T16:22:07.435416+0000","last_became_active":"2024-04-02T16:22:01.402419+0000","last_became_peered":"2024-04-02T16:22:01.402419+0000","last_unstale":"2024-04-02T16:22:07.435416+0000","last_undegraded":"2024-04-02T16:22:07.435416+0000","last_fullsized":"2024-04-02T16:22:07.435416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428475+0000","last_change":"2024-04-02T16:22:01.411196+0000","last_active":"2024-04-02T16:22:07.428475+0000","last_peered":"2024-04-02T16:22:07.428475+0000","last_clean":"2024-04-02T16:22:07.428475+0000","last_became_active":"2024-04-02T16:22:01.410630+0000","last_became_peered":"2024-04-02T16:22:01.410630+0000","last_unstale":"2024-04-02T16:22:07.428475+0000","last_undegraded":"2024-04-02T16:22:07.428475+0000","last_fullsized":"2024-04-02T16:22:07.428475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434802+0000","last_change":"2024-04-02T16:22:01.401113+0000","last_active":"2024-04-02T16:22:07.434802+0000","last_peered":"2024-04-02T16:22:07.434802+0000","last_clean":"2024-04-02T16:22:07.434802+0000","last_became_active":"2024-04-02T16:22:01.400823+0000","last_became_peered":"2024-04-02T16:22:01.400823+0000","last_unstale":"2024-04-02T16:22:07.434802+0000","last_undegraded":"2024-04-02T16:22:07.434802+0000","last_fullsized":"2024-04-02T16:22:07.434802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435248+0000","last_change":"2024-04-02T16:22:01.401273+0000","last_active":"2024-04-02T16:22:07.435248+0000","last_peered":"2024-04-02T16:22:07.435248+0000","last_clean":"2024-04-02T16:22:07.435248+0000","last_became_active":"2024-04-02T16:22:01.400989+0000","last_became_peered":"2024-04-02T16:22:01.400989+0000","last_unstale":"2024-04-02T16:22:07.435248+0000","last_undegraded":"2024-04-02T16:22:07.435248+0000","last_fullsized":"2024-04-02T16:22:07.435248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428640+0000","last_change":"2024-04-02T16:22:01.410167+0000","last_active":"2024-04-02T16:22:07.428640+0000","last_peered":"2024-04-02T16:22:07.428640+0000","last_clean":"2024-04-02T16:22:07.428640+0000","last_became_active":"2024-04-02T16:22:01.409747+0000","last_became_peered":"2024-04-02T16:22:01.409747+0000","last_unstale":"2024-04-02T16:22:07.428640+0000","last_undegraded":"2024-04-02T16:22:07.428640+0000","last_fullsized":"2024-04-02T16:22:07.428640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430485+0000","last_change":"2024-04-02T16:22:01.404722+0000","last_active":"2024-04-02T16:22:07.430485+0000","last_peered":"2024-04-02T16:22:07.430485+0000","last_clean":"2024-04-02T16:22:07.430485+0000","last_became_active":"2024-04-02T16:22:01.403645+0000","last_became_peered":"2024-04-02T16:22:01.403645+0000","last_unstale":"2024-04-02T16:22:07.430485+0000","last_undegraded":"2024-04-02T16:22:07.430485+0000","last_fullsized":"2024-04-02T16:22:07.430485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427877+0000","last_change":"2024-04-02T16:22:01.410096+0000","last_active":"2024-04-02T16:22:07.427877+0000","last_peered":"2024-04-02T16:22:07.427877+0000","last_clean":"2024-04-02T16:22:07.427877+0000","last_became_active":"2024-04-02T16:22:01.409805+0000","last_became_peered":"2024-04-02T16:22:01.409805+0000","last_unstale":"2024-04-02T16:22:07.427877+0000","last_undegraded":"2024-04-02T16:22:07.427877+0000","last_fullsized":"2024-04-02T16:22:07.427877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429582+0000","last_change":"2024-04-02T16:22:01.410138+0000","last_active":"2024-04-02T16:22:07.429582+0000","last_peered":"2024-04-02T16:22:07.429582+0000","last_clean":"2024-04-02T16:22:07.429582+0000","last_became_active":"2024-04-02T16:22:01.403179+0000","last_became_peered":"2024-04-02T16:22:01.403179+0000","last_unstale":"2024-04-02T16:22:07.429582+0000","last_undegraded":"2024-04-02T16:22:07.429582+0000","last_fullsized":"2024-04-02T16:22:07.429582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435300+0000","last_change":"2024-04-02T16:22:01.403457+0000","last_active":"2024-04-02T16:22:07.435300+0000","last_peered":"2024-04-02T16:22:07.435300+0000","last_clean":"2024-04-02T16:22:07.435300+0000","last_became_active":"2024-04-02T16:22:01.402335+0000","last_became_peered":"2024-04-02T16:22:01.402335+0000","last_unstale":"2024-04-02T16:22:07.435300+0000","last_undegraded":"2024-04-02T16:22:07.435300+0000","last_fullsized":"2024-04-02T16:22:07.435300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434948+0000","last_change":"2024-04-02T16:22:01.406544+0000","last_active":"2024-04-02T16:22:07.434948+0000","last_peered":"2024-04-02T16:22:07.434948+0000","last_clean":"2024-04-02T16:22:07.434948+0000","last_became_active":"2024-04-02T16:22:01.406205+0000","last_became_peered":"2024-04-02T16:22:01.406205+0000","last_unstale":"2024-04-02T16:22:07.434948+0000","last_undegraded":"2024-04-02T16:22:07.434948+0000","last_fullsized":"2024-04-02T16:22:07.434948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435085+0000","last_change":"2024-04-02T16:22:01.401972+0000","last_active":"2024-04-02T16:22:07.435085+0000","last_peered":"2024-04-02T16:22:07.435085+0000","last_clean":"2024-04-02T16:22:07.435085+0000","last_became_active":"2024-04-02T16:22:01.401721+0000","last_became_peered":"2024-04-02T16:22:01.401721+0000","last_unstale":"2024-04-02T16:22:07.435085+0000","last_undegraded":"2024-04-02T16:22:07.435085+0000","last_fullsized":"2024-04-02T16:22:07.435085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:05.479052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433925+0000","last_change":"2024-04-02T16:22:01.406917+0000","last_active":"2024-04-02T16:22:07.433925+0000","last_peered":"2024-04-02T16:22:07.433925+0000","last_clean":"2024-04-02T16:22:07.433925+0000","last_became_active":"2024-04-02T16:22:01.406622+0000","last_became_peered":"2024-04-02T16:22:01.406622+0000","last_unstale":"2024-04-02T16:22:07.433925+0000","last_undegraded":"2024-04-02T16:22:07.433925+0000","last_fullsized":"2024-04-02T16:22:07.433925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430970+0000","last_change":"2024-04-02T16:22:01.403891+0000","last_active":"2024-04-02T16:22:07.430970+0000","last_peered":"2024-04-02T16:22:07.430970+0000","last_clean":"2024-04-02T16:22:07.430970+0000","last_became_active":"2024-04-02T16:22:01.403250+0000","last_became_peered":"2024-04-02T16:22:01.403250+0000","last_unstale":"2024-04-02T16:22:07.430970+0000","last_undegraded":"2024-04-02T16:22:07.430970+0000","last_fullsized":"2024-04-02T16:22:07.430970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430459+0000","last_change":"2024-04-02T16:22:01.410686+0000","last_active":"2024-04-02T16:22:07.430459+0000","last_peered":"2024-04-02T16:22:07.430459+0000","last_clean":"2024-04-02T16:22:07.430459+0000","last_became_active":"2024-04-02T16:22:01.410398+0000","last_became_peered":"2024-04-02T16:22:01.410398+0000","last_unstale":"2024-04-02T16:22:07.430459+0000","last_undegraded":"2024-04-02T16:22:07.430459+0000","last_fullsized":"2024-04-02T16:22:07.430459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.406912+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.406626+0000","last_became_peered":"2024-04-02T16:22:01.406626+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434095+0000","last_change":"2024-04-02T16:22:01.397661+0000","last_active":"2024-04-02T16:22:07.434095+0000","last_peered":"2024-04-02T16:22:07.434095+0000","last_clean":"2024-04-02T16:22:07.434095+0000","last_became_active":"2024-04-02T16:22:01.397335+0000","last_became_peered":"2024-04-02T16:22:01.397335+0000","last_unstale":"2024-04-02T16:22:07.434095+0000","last_undegraded":"2024-04-02T16:22:07.434095+0000","last_fullsized":"2024-04-02T16:22:07.434095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428147+0000","last_change":"2024-04-02T16:22:01.409121+0000","last_active":"2024-04-02T16:22:07.428147+0000","last_peered":"2024-04-02T16:22:07.428147+0000","last_clean":"2024-04-02T16:22:07.428147+0000","last_became_active":"2024-04-02T16:22:01.408765+0000","last_became_peered":"2024-04-02T16:22:01.408765+0000","last_unstale":"2024-04-02T16:22:07.428147+0000","last_undegraded":"2024-04-02T16:22:07.428147+0000","last_fullsized":"2024-04-02T16:22:07.428147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429788+0000","last_change":"2024-04-02T16:22:01.407108+0000","last_active":"2024-04-02T16:22:07.429788+0000","last_peered":"2024-04-02T16:22:07.429788+0000","last_clean":"2024-04-02T16:22:07.429788+0000","last_became_active":"2024-04-02T16:22:01.406836+0000","last_became_peered":"2024-04-02T16:22:01.406836+0000","last_unstale":"2024-04-02T16:22:07.429788+0000","last_undegraded":"2024-04-02T16:22:07.429788+0000","last_fullsized":"2024-04-02T16:22:07.429788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434905+0000","last_change":"2024-04-02T16:22:01.414174+0000","last_active":"2024-04-02T16:22:07.434905+0000","last_peered":"2024-04-02T16:22:07.434905+0000","last_clean":"2024-04-02T16:22:07.434905+0000","last_became_active":"2024-04-02T16:22:01.413903+0000","last_became_peered":"2024-04-02T16:22:01.413903+0000","last_unstale":"2024-04-02T16:22:07.434905+0000","last_undegraded":"2024-04-02T16:22:07.434905+0000","last_fullsized":"2024-04-02T16:22:07.434905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435234+0000","last_change":"2024-04-02T16:22:01.413533+0000","last_active":"2024-04-02T16:22:07.435234+0000","last_peered":"2024-04-02T16:22:07.435234+0000","last_clean":"2024-04-02T16:22:07.435234+0000","last_became_active":"2024-04-02T16:22:01.413266+0000","last_became_peered":"2024-04-02T16:22:01.413266+0000","last_unstale":"2024-04-02T16:22:07.435234+0000","last_undegraded":"2024-04-02T16:22:07.435234+0000","last_fullsized":"2024-04-02T16:22:07.435234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435468+0000","last_change":"2024-04-02T16:22:01.412176+0000","last_active":"2024-04-02T16:22:07.435468+0000","last_peered":"2024-04-02T16:22:07.435468+0000","last_clean":"2024-04-02T16:22:07.435468+0000","last_became_active":"2024-04-02T16:22:01.411793+0000","last_became_peered":"2024-04-02T16:22:01.411793+0000","last_unstale":"2024-04-02T16:22:07.435468+0000","last_undegraded":"2024-04-02T16:22:07.435468+0000","last_fullsized":"2024-04-02T16:22:07.435468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:12.015838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427771+0000","last_change":"2024-04-02T16:22:01.410605+0000","last_active":"2024-04-02T16:22:07.427771+0000","last_peered":"2024-04-02T16:22:07.427771+0000","last_clean":"2024-04-02T16:22:07.427771+0000","last_became_active":"2024-04-02T16:22:01.410237+0000","last_became_peered":"2024-04-02T16:22:01.410237+0000","last_unstale":"2024-04-02T16:22:07.427771+0000","last_undegraded":"2024-04-02T16:22:07.427771+0000","last_fullsized":"2024-04-02T16:22:07.427771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434564+0000","last_change":"2024-04-02T16:22:01.404165+0000","last_active":"2024-04-02T16:22:07.434564+0000","last_peered":"2024-04-02T16:22:07.434564+0000","last_clean":"2024-04-02T16:22:07.434564+0000","last_became_active":"2024-04-02T16:22:01.403907+0000","last_became_peered":"2024-04-02T16:22:01.403907+0000","last_unstale":"2024-04-02T16:22:07.434564+0000","last_undegraded":"2024-04-02T16:22:07.434564+0000","last_fullsized":"2024-04-02T16:22:07.434564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429480+0000","last_change":"2024-04-02T16:22:01.409810+0000","last_active":"2024-04-02T16:22:07.429480+0000","last_peered":"2024-04-02T16:22:07.429480+0000","last_clean":"2024-04-02T16:22:07.429480+0000","last_became_active":"2024-04-02T16:22:01.406447+0000","last_became_peered":"2024-04-02T16:22:01.406447+0000","last_unstale":"2024-04-02T16:22:07.429480+0000","last_undegraded":"2024-04-02T16:22:07.429480+0000","last_fullsized":"2024-04-02T16:22:07.429480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434648+0000","last_change":"2024-04-02T16:22:01.401898+0000","last_active":"2024-04-02T16:22:07.434648+0000","last_peered":"2024-04-02T16:22:07.434648+0000","last_clean":"2024-04-02T16:22:07.434648+0000","last_became_active":"2024-04-02T16:22:01.401568+0000","last_became_peered":"2024-04-02T16:22:01.401568+0000","last_unstale":"2024-04-02T16:22:07.434648+0000","last_undegraded":"2024-04-02T16:22:07.434648+0000","last_fullsized":"2024-04-02T16:22:07.434648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430221+0000","last_change":"2024-04-02T16:22:01.410551+0000","last_active":"2024-04-02T16:22:07.430221+0000","last_peered":"2024-04-02T16:22:07.430221+0000","last_clean":"2024-04-02T16:22:07.430221+0000","last_became_active":"2024-04-02T16:22:01.409045+0000","last_became_peered":"2024-04-02T16:22:01.409045+0000","last_unstale":"2024-04-02T16:22:07.430221+0000","last_undegraded":"2024-04-02T16:22:07.430221+0000","last_fullsized":"2024-04-02T16:22:07.430221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434517+0000","last_change":"2024-04-02T16:22:01.398994+0000","last_active":"2024-04-02T16:22:07.434517+0000","last_peered":"2024-04-02T16:22:07.434517+0000","last_clean":"2024-04-02T16:22:07.434517+0000","last_became_active":"2024-04-02T16:22:01.398698+0000","last_became_peered":"2024-04-02T16:22:01.398698+0000","last_unstale":"2024-04-02T16:22:07.434517+0000","last_undegraded":"2024-04-02T16:22:07.434517+0000","last_fullsized":"2024-04-02T16:22:07.434517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435172+0000","last_change":"2024-04-02T16:22:01.406275+0000","last_active":"2024-04-02T16:22:07.435172+0000","last_peered":"2024-04-02T16:22:07.435172+0000","last_clean":"2024-04-02T16:22:07.435172+0000","last_became_active":"2024-04-02T16:22:01.405936+0000","last_became_peered":"2024-04-02T16:22:01.405936+0000","last_unstale":"2024-04-02T16:22:07.435172+0000","last_undegraded":"2024-04-02T16:22:07.435172+0000","last_fullsized":"2024-04-02T16:22:07.435172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435388+0000","last_change":"2024-04-02T16:22:01.407470+0000","last_active":"2024-04-02T16:22:07.435388+0000","last_peered":"2024-04-02T16:22:07.435388+0000","last_clean":"2024-04-02T16:22:07.435388+0000","last_became_active":"2024-04-02T16:22:01.407209+0000","last_became_peered":"2024-04-02T16:22:01.407209+0000","last_unstale":"2024-04-02T16:22:07.435388+0000","last_undegraded":"2024-04-02T16:22:07.435388+0000","last_fullsized":"2024-04-02T16:22:07.435388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:31.303906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:22:24.847 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T16:22:24.847 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T16:22:24.847 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-02T16:22:24.848 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-02T16:22:24.848 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-02T16:22:24.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.982+0000 7f9a69304640 1 Processor -- start 2024-04-02T16:22:24.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.982+0000 7f9a69304640 1 -- start start 2024-04-02T16:22:24.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a69304640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a69304640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a640b4760 0x7f9a640d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a69304640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9a640d2cb0 0x7f9a641622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a69304640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f9a640b5880 con 0x7f9a640d2cb0 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a69304640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9a640b4c70 con 0x7f9a640b4760 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a69304640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a640b4f50 con 0x7f9a6415e650 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a637fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9a640d2cb0 0x7f9a641622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a637fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9a640d2cb0 0x7f9a641622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40118/0 (socket says 172.21.15.50:40118) 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.983+0000 7f9a637fe640 1 -- 172.21.15.50:0/215368605 learned_addr learned my addr 172.21.15.50:0/215368605 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a627fc640 1 --2- 172.21.15.50:0/215368605 >> v2:172.21.15.188:3300/0 conn(0x7f9a640b4760 0x7f9a640d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a62ffd640 1 --2- 172.21.15.50:0/215368605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a637fe640 1 -- 172.21.15.50:0/215368605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 msgr2=0x7f9a640b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a637fe640 1 --2- 172.21.15.50:0/215368605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a637fe640 1 -- 172.21.15.50:0/215368605 >> v2:172.21.15.188:3300/0 conn(0x7f9a640b4760 msgr2=0x7f9a640d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a637fe640 1 --2- 172.21.15.50:0/215368605 >> v2:172.21.15.188:3300/0 conn(0x7f9a640b4760 0x7f9a640d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a637fe640 1 -- 172.21.15.50:0/215368605 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a640b5230 con 0x7f9a640d2cb0 2024-04-02T16:22:24.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a627fc640 1 --2- 172.21.15.50:0/215368605 >> v2:172.21.15.188:3300/0 conn(0x7f9a640b4760 0x7f9a640d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:24.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.984+0000 7f9a637fe640 1 --2- 172.21.15.50:0/215368605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a641622e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a6408df00 tx=0x7f9a5402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19288d730644612 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a61ffb640 1 -- 172.21.15.50:0/215368605 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5402ca70 con 0x7f9a640d2cb0 2024-04-02T16:22:24.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a61ffb640 1 -- 172.21.15.50:0/215368605 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5402cc10 con 0x7f9a640d2cb0 2024-04-02T16:22:24.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a61ffb640 1 -- 172.21.15.50:0/215368605 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540343f0 con 0x7f9a640d2cb0 2024-04-02T16:22:24.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 -- 172.21.15.50:0/215368605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 msgr2=0x7f9a641622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 --2- 172.21.15.50:0/215368605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a641622e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a6408df00 tx=0x7f9a5402bed0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 -- 172.21.15.50:0/215368605 shutdown_connections 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 --2- 172.21.15.50:0/215368605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a641622e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 --2- 172.21.15.50:0/215368605 >> v2:172.21.15.188:3300/0 conn(0x7f9a640b4760 0x7f9a640d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 --2- 172.21.15.50:0/215368605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 -- 172.21.15.50:0/215368605 >> 172.21.15.50:0/215368605 conn(0x7f9a6409d580 msgr2=0x7f9a640d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 -- 172.21.15.50:0/215368605 shutdown_connections 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.985+0000 7f9a69304640 1 -- 172.21.15.50:0/215368605 wait complete. 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.986+0000 7f9a69304640 1 Processor -- start 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a69304640 1 -- start start 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a69304640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a640b4760 0x7f9a6414f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a69304640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a6415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a69304640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a69304640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6415ea30 con 0x7f9a640d2cb0 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a69304640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a640d2270 con 0x7f9a640b4760 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a69304640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6415ebb0 con 0x7f9a6415e650 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a627fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a6415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a627fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a6415f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40134/0 (socket says 172.21.15.50:40134) 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a627fc640 1 -- 172.21.15.50:0/2992933430 learned_addr learned my addr 172.21.15.50:0/2992933430 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a637fe640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.987+0000 7f9a62ffd640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a640b4760 0x7f9a6414f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a627fc640 1 -- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 msgr2=0x7f9a640b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a627fc640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a627fc640 1 -- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a640b4760 msgr2=0x7f9a6414f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a627fc640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a640b4760 0x7f9a6414f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a627fc640 1 -- 172.21.15.50:0/2992933430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a640b6e70 con 0x7f9a640d2cb0 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a627fc640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a6415f7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c7c0 tx=0x7f9a4c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a5b7fe640 1 -- 172.21.15.50:0/2992933430 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c015070 con 0x7f9a640d2cb0 2024-04-02T16:22:24.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a5b7fe640 1 -- 172.21.15.50:0/2992933430 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0077c0 con 0x7f9a640d2cb0 2024-04-02T16:22:24.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a64160210 con 0x7f9a640d2cb0 2024-04-02T16:22:24.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a5b7fe640 1 -- 172.21.15.50:0/2992933430 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c007b90 con 0x7f9a640d2cb0 2024-04-02T16:22:24.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.988+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a640b7110 con 0x7f9a640d2cb0 2024-04-02T16:22:24.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.989+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a640756f0 con 0x7f9a640d2cb0 2024-04-02T16:22:24.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.989+0000 7f9a5b7fe640 1 -- 172.21.15.50:0/2992933430 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a4c018070 con 0x7f9a640d2cb0 2024-04-02T16:22:24.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.990+0000 7f9a5b7fe640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a4406c8f0 0x7f9a4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:24.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.990+0000 7f9a5b7fe640 1 -- 172.21.15.50:0/2992933430 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9a4c08bc40 con 0x7f9a640d2cb0 2024-04-02T16:22:24.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.990+0000 7f9a62ffd640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a4406c8f0 0x7f9a4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:24.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.991+0000 7f9a62ffd640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a4406c8f0 0x7f9a4406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00a9a0 tx=0x7f9a5c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:24.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:24.994+0000 7f9a5b7fe640 1 -- 172.21.15.50:0/2992933430 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9a4c0596e0 con 0x7f9a640d2cb0 2024-04-02T16:22:25.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.232+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9a640d1aa0 con 0x7f9a640d2cb0 2024-04-02T16:22:25.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.235+0000 7f9a5b7fe640 1 -- 172.21.15.50:0/2992933430 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174141 (secure 0 0 0) 0x7f9a4c05d9e0 con 0x7f9a640d2cb0 2024-04-02T16:22:25.239 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:25.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.237+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a4406c8f0 msgr2=0x7f9a4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.237+0000 7f9a69304640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a4406c8f0 0x7f9a4406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00a9a0 tx=0x7f9a5c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:25.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.237+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 msgr2=0x7f9a6415f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.237+0000 7f9a69304640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a6415f7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c7c0 tx=0x7f9a4c00cc80 comp rx=0 tx=0).stop 2024-04-02T16:22:25.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 shutdown_connections 2024-04-02T16:22:25.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a6415e650 0x7f9a640b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a4406c8f0 0x7f9a4406edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a640d2cb0 0x7f9a6415f7c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 --2- 172.21.15.50:0/2992933430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a640b4760 0x7f9a6414f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 >> 172.21.15.50:0/2992933430 conn(0x7f9a6409d580 msgr2=0x7f9a640d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 shutdown_connections 2024-04-02T16:22:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.238+0000 7f9a69304640 1 -- 172.21.15.50:0/2992933430 wait complete. 2024-04-02T16:22:25.249 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":5,"flags":0,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2785600878},{"type":"v1","addr":"172.21.15.188:6833","nonce":2785600878}]},"active_addr":"172.21.15.188:6833/2785600878","active_change":"2024-04-02T16:21:40.657293+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":559212792}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":790142891}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":2069252280}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":478009820}]}]} 2024-04-02T16:22:25.252 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-02T16:22:25.253 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-02T16:22:25.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:25.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.353+0000 7fbc04ae4640 1 Processor -- start 2024-04-02T16:22:25.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.353+0000 7fbc04ae4640 1 -- start start 2024-04-02T16:22:25.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbc04ae4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbc000b0790 0x7fbc000b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbc04ae4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc000724c0 con 0x7fbc0015e590 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbc04ae4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc0006c1d0 con 0x7fbc0006bcf0 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbc04ae4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fbc0006c4b0 con 0x7fbc000b0790 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbbfdd9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbc000b0790 0x7fbc000b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbbfe59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fbc0015e590 0x7fbc000ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47146/0 (socket says 172.21.15.50:47146) 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbbfe59c640 1 -- 172.21.15.50:0/773898530 learned_addr learned my addr 172.21.15.50:0/773898530 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbbfdd9b640 1 -- 172.21.15.50:0/773898530 >> v1:172.21.15.188:6789/0 conn(0x7fbc0006bcf0 legacy=0x7fbc0006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbbfdd9b640 1 -- 172.21.15.50:0/773898530 >> v1:172.21.15.50:6789/0 conn(0x7fbc0015e590 legacy=0x7fbc000ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbbfd59a640 1 -- 172.21.15.50:0/773898530 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1365722353 0 0) 0x7fbc000724c0 con 0x7fbc0015e590 2024-04-02T16:22:25.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.354+0000 7fbbfdd9b640 1 -- 172.21.15.50:0/773898530 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc00072760 con 0x7fbc000b0790 2024-04-02T16:22:25.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.355+0000 7fbbfdd9b640 1 --2- 172.21.15.50:0/773898530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc000b0790 0x7fbc000b49e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40097a0 tx=0x7fbbf402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdb350b8384a443 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.355+0000 7fbbfd59a640 1 -- 172.21.15.50:0/773898530 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4035020 con 0x7fbc000b0790 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.355+0000 7fbbfd59a640 1 -- 172.21.15.50:0/773898530 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf4031830 con 0x7fbc000b0790 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.355+0000 7fbbfd59a640 1 -- 172.21.15.50:0/773898530 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4031c00 con 0x7fbc000b0790 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.356+0000 7fbc04ae4640 1 -- 172.21.15.50:0/773898530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc000b0790 msgr2=0x7fbc000b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.356+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/773898530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc000b0790 0x7fbc000b49e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40097a0 tx=0x7fbbf402b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.356+0000 7fbc04ae4640 1 -- 172.21.15.50:0/773898530 shutdown_connections 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.356+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/773898530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc000b0790 0x7fbc000b49e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.356+0000 7fbc04ae4640 1 -- 172.21.15.50:0/773898530 >> 172.21.15.50:0/773898530 conn(0x7fbc000b6540 msgr2=0x7fbc000b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.356+0000 7fbc04ae4640 1 -- 172.21.15.50:0/773898530 shutdown_connections 2024-04-02T16:22:25.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.356+0000 7fbc04ae4640 1 -- 172.21.15.50:0/773898530 wait complete. 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.357+0000 7fbc04ae4640 1 Processor -- start 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.357+0000 7fbc04ae4640 1 -- start start 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbc04ae4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0006bcf0 0x7fbc00161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbc04ae4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc000b0790 0x7fbc0014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbc04ae4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc0015e590 0x7fbc000b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbc04ae4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000adfc0 con 0x7fbc0006bcf0 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbc04ae4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0015e970 con 0x7fbc0015e590 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbc04ae4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0006f080 con 0x7fbc000b0790 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0006bcf0 0x7fbc00161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0006bcf0 0x7fbc00161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40148/0 (socket says 172.21.15.50:40148) 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 -- 172.21.15.50:0/1182961486 learned_addr learned my addr 172.21.15.50:0/1182961486 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfed9d640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc0015e590 0x7fbc000b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfdd9b640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc000b0790 0x7fbc0014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 -- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc000b0790 msgr2=0x7fbc0014aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc000b0790 0x7fbc0014aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 -- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc0015e590 msgr2=0x7fbc000b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc0015e590 0x7fbc000b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 -- 172.21.15.50:0/1182961486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc00151120 con 0x7fbc0006bcf0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.358+0000 7fbbfe59c640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0006bcf0 0x7fbc00161300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c3c0 tx=0x7fbbf000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.359+0000 7fbbeeffd640 1 -- 172.21.15.50:0/1182961486 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0017070 con 0x7fbc0006bcf0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.359+0000 7fbbeeffd640 1 -- 172.21.15.50:0/1182961486 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf0005ce0 con 0x7fbc0006bcf0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.359+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc000cd630 con 0x7fbc0006bcf0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.359+0000 7fbbeeffd640 1 -- 172.21.15.50:0/1182961486 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0007310 con 0x7fbc0006bcf0 2024-04-02T16:22:25.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.359+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0014f190 con 0x7fbc0006bcf0 2024-04-02T16:22:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.360+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc000b5010 con 0x7fbc0006bcf0 2024-04-02T16:22:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.360+0000 7fbbeeffd640 1 -- 172.21.15.50:0/1182961486 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbf001a070 con 0x7fbc0006bcf0 2024-04-02T16:22:25.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.360+0000 7fbbeeffd640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbbe006c8f0 0x7fbbe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.360+0000 7fbbeeffd640 1 -- 172.21.15.50:0/1182961486 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbbf008b220 con 0x7fbc0006bcf0 2024-04-02T16:22:25.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.361+0000 7fbbfdd9b640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbbe006c8f0 0x7fbbe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.361+0000 7fbbfdd9b640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbbe006c8f0 0x7fbbe006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4036000 tx=0x7fbbf4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:25.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.365+0000 7fbbeeffd640 1 -- 172.21.15.50:0/1182961486 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbc000b5010 con 0x7fbc0006bcf0 2024-04-02T16:22:25.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.554+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc000756f0 con 0x7fbc0006bcf0 2024-04-02T16:22:25.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.555+0000 7fbbeeffd640 1 -- 172.21.15.50:0/1182961486 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fbbf0059310 con 0x7fbc0006bcf0 2024-04-02T16:22:25.557 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:25.557 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbbe006c8f0 msgr2=0x7fbbe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbbe006c8f0 0x7fbbe006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4036000 tx=0x7fbbf4008030 comp rx=0 tx=0).stop 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0006bcf0 msgr2=0x7fbc00161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0006bcf0 0x7fbc00161300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c3c0 tx=0x7fbbf000c880 comp rx=0 tx=0).stop 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 shutdown_connections 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc0015e590 0x7fbc000b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc000b0790 0x7fbc0014aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbbe006c8f0 0x7fbbe006edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 --2- 172.21.15.50:0/1182961486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc0006bcf0 0x7fbc00161300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 >> 172.21.15.50:0/1182961486 conn(0x7fbc000b6540 msgr2=0x7fbc0015d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 shutdown_connections 2024-04-02T16:22:25.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.557+0000 7fbc04ae4640 1 -- 172.21.15.50:0/1182961486 wait complete. 2024-04-02T16:22:25.569 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-02T16:22:25.570 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:25.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.670+0000 7f58a7da8640 1 Processor -- start 2024-04-02T16:22:25.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.670+0000 7f58a7da8640 1 -- start start 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a7da8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 0x7f58a00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a7da8640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a00ceda0 con 0x7f58a015e510 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a7da8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a00cf040 con 0x7f58a00d0e90 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a7da8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f58a00cf320 con 0x7f58a00d2f10 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a5b44640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f58a015e510 0x7f58a00d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:54958/0 (socket says 172.21.15.50:54958) 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a5b44640 1 -- 172.21.15.50:0/2816680306 learned_addr learned my addr 172.21.15.50:0/2816680306 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585817667 0 0) 0x7f58a00ceda0 con 0x7f58a015e510 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f588c003520 con 0x7f58a015e510 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a5343640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 0x7f58a00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 245520023 0 0) 0x7f588c003520 con 0x7f58a015e510 2024-04-02T16:22:25.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 msgr2=0x7f58a00cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 0x7f58a00cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 >> v1:172.21.15.188:6789/0 conn(0x7f58a00d0e90 legacy=0x7f58a00d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a00cf600 con 0x7f58a015e510 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f589c001670 con 0x7f58a015e510 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 >> v1:172.21.15.50:6789/0 conn(0x7f58a015e510 legacy=0x7f58a00d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f588c0039e0 0x7f588c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c004800 0x7f588c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 0x7f588c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00cf320 con 0x7f588c0039e0 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f588c00bb50 con 0x7f588c004800 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f588c00bd50 con 0x7f58a00d2f10 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.671+0000 7f58a5b44640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f588c0039e0 0x7f588c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a5343640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c004800 0x7f588c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a6345640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 0x7f588c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a5b44640 1 -- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 msgr2=0x7f588c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a5b44640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00d2f10 0x7f588c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a5b44640 1 -- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c004800 msgr2=0x7f588c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a5b44640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c004800 0x7f588c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a5b44640 1 -- 172.21.15.50:0/2816680306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c003520 con 0x7f588c0039e0 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.672+0000 7f58a5b44640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f588c0039e0 0x7f588c003e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c190 tx=0x7f588c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa314418683b92f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c007c60 con 0x7f588c0039e0 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589c030070 con 0x7f588c0039e0 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a4b42640 1 -- 172.21.15.50:0/2816680306 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c005d70 con 0x7f588c0039e0 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a7da8640 1 -- 172.21.15.50:0/2816680306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f588c0039e0 msgr2=0x7f588c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a7da8640 1 --2- 172.21.15.50:0/2816680306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f588c0039e0 0x7f588c003e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c190 tx=0x7f588c005830 comp rx=0 tx=0).stop 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a6345640 1 -- 172.21.15.50:0/2816680306 reap_dead start 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a7da8640 1 -- 172.21.15.50:0/2816680306 shutdown_connections 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a7da8640 1 -- 172.21.15.50:0/2816680306 >> 172.21.15.50:0/2816680306 conn(0x7f58a009d570 msgr2=0x7f58a00d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:25.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a7da8640 1 -- 172.21.15.50:0/2816680306 shutdown_connections 2024-04-02T16:22:25.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.673+0000 7f58a7da8640 1 -- 172.21.15.50:0/2816680306 wait complete. 2024-04-02T16:22:25.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.674+0000 7f58a7da8640 1 Processor -- start 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.674+0000 7f58a7da8640 1 -- start start 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a7da8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c0039e0 0x7f58a015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a7da8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00b1b80 0x7f58a00cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a7da8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58a00cb8a0 0x7f58a015f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a7da8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00cff80 con 0x7f58a00cb8a0 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a7da8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00cfe00 con 0x7f588c0039e0 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a7da8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f58a00d0100 con 0x7f58a00b1b80 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a6345640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58a00cb8a0 0x7f58a015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a6345640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58a00cb8a0 0x7f58a015f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44914/0 (socket says 172.21.15.50:44914) 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a6345640 1 -- 172.21.15.50:0/1769260762 learned_addr learned my addr 172.21.15.50:0/1769260762 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a5b44640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c0039e0 0x7f58a015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.675+0000 7f58a5343640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00b1b80 0x7f58a00cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a6345640 1 -- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00b1b80 msgr2=0x7f58a00cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a6345640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00b1b80 0x7f58a00cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a6345640 1 -- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c0039e0 msgr2=0x7f58a015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a6345640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c0039e0 0x7f58a015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a6345640 1 -- 172.21.15.50:0/1769260762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a00cbed0 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a6345640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58a00cb8a0 0x7f58a015f960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58a00b5d50 tx=0x7f589800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58967fc640 1 -- 172.21.15.50:0/1769260762 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589801a070 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58967fc640 1 -- 172.21.15.50:0/1769260762 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5898006d60 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a00cc1b0 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58967fc640 1 -- 172.21.15.50:0/1769260762 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898003b90 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.676+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a00cc770 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.677+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a00756f0 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.678+0000 7f58967fc640 1 -- 172.21.15.50:0/1769260762 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f589801d070 con 0x7f58a00cb8a0 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.678+0000 7f58967fc640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f588406c920 0x7f588406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:25.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.678+0000 7f58967fc640 1 -- 172.21.15.50:0/1769260762 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f589808b320 con 0x7f58a00cb8a0 2024-04-02T16:22:25.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.678+0000 7f58a5b44640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f588406c920 0x7f588406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:25.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.679+0000 7f58a5b44640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f588406c920 0x7f588406ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f589c034000 tx=0x7f589c009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:25.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.682+0000 7f58967fc640 1 -- 172.21.15.50:0/1769260762 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5898058e50 con 0x7f58a00cb8a0 2024-04-02T16:22:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.868+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58a006dad0 con 0x7f58a00cb8a0 2024-04-02T16:22:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.869+0000 7f58967fc640 1 -- 172.21.15.50:0/1769260762 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f589805d150 con 0x7f58a00cb8a0 2024-04-02T16:22:25.871 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:25.871 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f588406c920 msgr2=0x7f588406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f588406c920 0x7f588406ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f589c034000 tx=0x7f589c009030 comp rx=0 tx=0).stop 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58a00cb8a0 msgr2=0x7f58a015f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58a00cb8a0 0x7f58a015f960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58a00b5d50 tx=0x7f589800fc80 comp rx=0 tx=0).stop 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 shutdown_connections 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f588406c920 0x7f588406ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58a00cb8a0 0x7f58a015f960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58a00b1b80 0x7f58a00cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 --2- 172.21.15.50:0/1769260762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c0039e0 0x7f58a015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 >> 172.21.15.50:0/1769260762 conn(0x7f58a009d570 msgr2=0x7f58a00d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:25.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 shutdown_connections 2024-04-02T16:22:25.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:25.871+0000 7f58a7da8640 1 -- 172.21.15.50:0/1769260762 wait complete. 2024-04-02T16:22:25.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-02T16:22:25.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-02T16:22:25.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-02T16:22:25.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-02T16:22:25.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-02T16:22:25.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-02T16:22:25.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-02T16:22:25.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-02T16:22:26.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.059+0000 7f18ddcaf640 1 Processor -- start 2024-04-02T16:22:26.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.059+0000 7f18ddcaf640 1 -- start start 2024-04-02T16:22:26.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.059+0000 7f95510b1640 1 Processor -- start 2024-04-02T16:22:26.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.059+0000 7f95510b1640 1 -- start start 2024-04-02T16:22:26.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.059+0000 7fa892ea0640 1 Processor -- start 2024-04-02T16:22:26.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.059+0000 7fa892ea0640 1 -- start start 2024-04-02T16:22:26.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7fe024805640 1 Processor -- start 2024-04-02T16:22:26.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f67994dd640 1 Processor -- start 2024-04-02T16:22:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7fe024805640 1 -- start start 2024-04-02T16:22:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f18ddcaf640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f18d80c55d0 0x7f18d80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f18ddcaf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18d8166c10 0x7f18d80af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18ddcaf640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d80aedc0 con 0x7f18d8162840 2024-04-02T16:22:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18ddcaf640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f18d81670a0 con 0x7f18d80c55d0 2024-04-02T16:22:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18ddcaf640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f18d806c760 con 0x7f18d8166c10 2024-04-02T16:22:26.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f95510b1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c13fe00 0x7f954c13d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f95510b1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f954c0cfea0 0x7f954c0748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f95510b1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f954c13ddb0 0x7f954c071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f95510b1640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f954c15ac70 con 0x7f954c13ddb0 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f95510b1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f954c1403e0 con 0x7f954c0cfea0 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f95510b1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f954c1406c0 con 0x7f954c13fe00 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.059+0000 7f3b6c48f640 1 Processor -- start 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f3b6c48f640 1 -- start start 2024-04-02T16:22:26.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f05d9af2640 1 Processor -- start 2024-04-02T16:22:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f05d9af2640 1 -- start start 2024-04-02T16:22:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7f67994dd640 1 -- start start 2024-04-02T16:22:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7fa892ea0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c15b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7fa892ea0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa88c0b31d0 0x7fa88c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7fa892ea0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa88c0719a0 con 0x7fa88c15d7c0 2024-04-02T16:22:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7fa892ea0640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fa88c15dba0 con 0x7fa88c0b31d0 2024-04-02T16:22:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.060+0000 7fa892ea0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa88c071c80 con 0x7fa88c0d0930 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fa89143d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa88c0b31d0 0x7fa88c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fa89143d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa88c0b31d0 0x7fa88c0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44916/0 (socket says 172.21.15.50:44916) 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fa89143d640 1 -- 172.21.15.50:0/1154552177 learned_addr learned my addr 172.21.15.50:0/1154552177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fe024805640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180e7820 0x7fe018038280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fe024805640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe0180061a0 0x7fe018006630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fe024805640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0180e7c00 con 0x7fe0180eb300 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fe024805640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fe018057f20 con 0x7fe0180061a0 2024-04-02T16:22:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fe024805640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe018058200 con 0x7fe0180e7820 2024-04-02T16:22:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c13fe00 0x7f954c13d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c13fe00 0x7f954c13d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57100/0 (socket says 172.21.15.50:57100) 2024-04-02T16:22:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 -- 172.21.15.50:0/378156963 learned_addr learned my addr 172.21.15.50:0/378156963 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18dd4ae640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18d8166c10 0x7f18d80af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18dccad640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f18d8162840 0x7f18d80c4ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:54968/0 (socket says 172.21.15.50:54968) 2024-04-02T16:22:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18dccad640 1 -- 172.21.15.50:0/2367346062 learned_addr learned my addr 172.21.15.50:0/2367346062 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fa88bfff640 1 --2- 172.21.15.50:0/1154552177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954a59c640 1 --2- 172.21.15.50:0/378156963 >> v2:172.21.15.188:3300/0 conn(0x7f954c0cfea0 0x7f954c0748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f05d9af2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f05d4145ae0 0x7f05d406d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f05d9af2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d415c700 con 0x7f05d415d460 2024-04-02T16:22:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f05d9af2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d4160a90 con 0x7f05d4160470 2024-04-02T16:22:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f05d9af2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f05d415cfd0 con 0x7f05d4145ae0 2024-04-02T16:22:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f67994dd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f679415c510 0x7f67941609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f67994dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67940b1890 0x7f67940cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f67994dd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67940cc230 con 0x7f679415c8f0 2024-04-02T16:22:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f67994dd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f679415d9b0 con 0x7f679415c510 2024-04-02T16:22:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f67994dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f67940cc4d0 con 0x7f67940b1890 2024-04-02T16:22:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18dd4ae640 1 -- 172.21.15.50:0/2367346062 >> v2:172.21.15.50:3300/0 conn(0x7f18d80c55d0 msgr2=0x7f18d80b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18d7fff640 1 --2- 172.21.15.50:0/2367346062 >> v2:172.21.15.50:3300/0 conn(0x7f18d80c55d0 0x7f18d80b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7fe01e59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe0180eb300 0x7fe0180eb780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:54982/0 (socket says 172.21.15.50:54982) 2024-04-02T16:22:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fe01e59c640 1 -- 172.21.15.50:0/2334920683 learned_addr learned my addr 172.21.15.50:0/2334920683 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f18dd4ae640 1 --2- 172.21.15.50:0/2367346062 >> v2:172.21.15.50:3300/0 conn(0x7f18d80c55d0 0x7f18d80b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f18dd4ae640 1 -- 172.21.15.50:0/2367346062 >> v1:172.21.15.50:6789/0 conn(0x7f18d8162840 legacy=0x7f18d80c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fe01ed9d640 1 --2- 172.21.15.50:0/2334920683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180e7820 0x7fe018038280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f67927fc640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f679415c8f0 0x7f67940b1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:44020/0 (socket says 172.21.15.50:44020) 2024-04-02T16:22:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f05d3fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f05d4160470 0x7f05d40b3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:44014/0 (socket says 172.21.15.50:44014) 2024-04-02T16:22:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d3fff640 1 -- 172.21.15.50:0/3858034566 learned_addr learned my addr 172.21.15.50:0/3858034566 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d37fe640 1 --2- 172.21.15.50:0/3858034566 >> v2:172.21.15.188:3300/0 conn(0x7f05d4145ae0 0x7f05d406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f67927fc640 1 -- 172.21.15.50:0/3216717582 learned_addr learned my addr 172.21.15.50:0/3216717582 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513895789 0 0) 0x7f05d4160a90 con 0x7f05d4160470 2024-04-02T16:22:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05bc003520 con 0x7f05d4160470 2024-04-02T16:22:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fe01f59e640 1 --2- 172.21.15.50:0/2334920683 >> v2:172.21.15.50:3300/0 conn(0x7fe0180061a0 0x7fe018006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fa88bfff640 1 -- 172.21.15.50:0/1154552177 >> v2:172.21.15.50:3300/0 conn(0x7fa88c0b31d0 msgr2=0x7fa88c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fa88bfff640 1 --2- 172.21.15.50:0/1154552177 >> v2:172.21.15.50:3300/0 conn(0x7fa88c0b31d0 0x7fa88c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fa88b7fe640 1 -- 172.21.15.50:0/1154552177 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 912202470 0 0) 0x7fa88c0719a0 con 0x7fa88c15d7c0 2024-04-02T16:22:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fa88bfff640 1 -- 172.21.15.50:0/1154552177 >> v1:172.21.15.188:6789/0 conn(0x7fa88c15d7c0 legacy=0x7fa88c0d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7fa88bfff640 1 -- 172.21.15.50:0/1154552177 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c071f60 con 0x7fa88c0d0930 2024-04-02T16:22:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861487498 0 0) 0x7f67940cc230 con 0x7f679415c8f0 2024-04-02T16:22:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f677c003520 con 0x7f679415c8f0 2024-04-02T16:22:26.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1579952128 0 0) 0x7f05bc003520 con 0x7f05d4160470 2024-04-02T16:22:26.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 >> v2:172.21.15.188:3300/0 conn(0x7f05d4145ae0 msgr2=0x7f05d406d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d27fc640 1 --2- 172.21.15.50:0/3858034566 >> v2:172.21.15.188:3300/0 conn(0x7f05d4145ae0 0x7f05d406d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 >> v1:172.21.15.50:6789/0 conn(0x7f05d415d460 legacy=0x7f05d415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7febc59cd640 1 Processor -- start 2024-04-02T16:22:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b6c48f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3b640b3d00 0x7f3b6406d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b6a22b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f3b640b3920 0x7f3b640b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:54990/0 (socket says 172.21.15.50:54990) 2024-04-02T16:22:26.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b6a22b640 1 -- 172.21.15.50:0/2178942378 learned_addr learned my addr 172.21.15.50:0/2178942378 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b6c48f640 1 --2- 172.21.15.50:0/2178942378 >> v2:172.21.15.188:3300/0 conn(0x7f3b64166fe0 0x7f3b6406d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b640b4e10 con 0x7f3b640b3920 2024-04-02T16:22:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f3b64166b50 con 0x7f3b640b3d00 2024-04-02T16:22:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3b64166500 con 0x7f3b64166fe0 2024-04-02T16:22:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f3b69a2a640 1 --2- 172.21.15.50:0/2178942378 >> v2:172.21.15.50:3300/0 conn(0x7f3b640b3d00 0x7f3b6406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7febc59cd640 1 -- start start 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 615005986 0 0) 0x7f677c003520 con 0x7f679415c8f0 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67940b1890 msgr2=0x7f67940cc9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67940b1890 0x7f67940cc9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 >> v2:172.21.15.50:3300/0 conn(0x7f679415c510 msgr2=0x7f67941609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 --2- 172.21.15.50:0/3216717582 >> v2:172.21.15.50:3300/0 conn(0x7f679415c510 0x7f67941609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679415d110 con 0x7f679415c8f0 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f6788001670 con 0x7f679415c8f0 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 >> v1:172.21.15.188:6789/0 conn(0x7f679415c8f0 legacy=0x7f67940b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f677c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f677c004800 0x7f677c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f677c005210 0x7f677c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f679415d9b0 con 0x7f677c005210 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67940cc4d0 con 0x7f677c0039d0 2024-04-02T16:22:26.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f677c00c190 con 0x7f677c004800 2024-04-02T16:22:26.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.062+0000 7f18dd4ae640 1 -- 172.21.15.50:0/2367346062 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d806ca40 con 0x7f18d8166c10 2024-04-02T16:22:26.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f18d7fff640 1 --2- 172.21.15.50:0/2367346062 >> v2:172.21.15.50:3300/0 conn(0x7f18d80c55d0 0x7f18d80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7fe01ed9d640 1 -- 172.21.15.50:0/2334920683 >> v2:172.21.15.50:3300/0 conn(0x7fe0180061a0 msgr2=0x7fe018006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7fe01ed9d640 1 --2- 172.21.15.50:0/2334920683 >> v2:172.21.15.50:3300/0 conn(0x7fe0180061a0 0x7fe018006630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7fe01ed9d640 1 -- 172.21.15.50:0/2334920683 >> v1:172.21.15.50:6789/0 conn(0x7fe0180eb300 legacy=0x7fe0180eb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7fe01ed9d640 1 -- 172.21.15.50:0/2334920683 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0180584e0 con 0x7fe0180e7820 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f3b69a2a640 1 -- 172.21.15.50:0/2178942378 >> v2:172.21.15.188:3300/0 conn(0x7f3b64166fe0 msgr2=0x7f3b6406d770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f3b69a2a640 1 --2- 172.21.15.50:0/2178942378 >> v2:172.21.15.188:3300/0 conn(0x7f3b64166fe0 0x7f3b6406d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f3b69a2a640 1 -- 172.21.15.50:0/2178942378 >> v1:172.21.15.50:6789/0 conn(0x7f3b640b3920 legacy=0x7f3b640b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f3b69a2a640 1 -- 172.21.15.50:0/2178942378 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b641667e0 con 0x7f3b640b3d00 2024-04-02T16:22:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f18dd4ae640 1 --2- 172.21.15.50:0/2367346062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 0x7f18d80af580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18d806ef60 tx=0x7f18cc02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9354ee398c76abca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 -- 172.21.15.50:0/378156963 >> v2:172.21.15.188:3300/0 conn(0x7f954c0cfea0 msgr2=0x7f954c0748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 --2- 172.21.15.50:0/378156963 >> v2:172.21.15.188:3300/0 conn(0x7f954c0cfea0 0x7f954c0748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 -- 172.21.15.50:0/378156963 >> v2:172.21.15.50:3300/0 conn(0x7f954c13ddb0 msgr2=0x7f954c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 --2- 172.21.15.50:0/378156963 >> v2:172.21.15.50:3300/0 conn(0x7f954c13ddb0 0x7f954c071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.061+0000 7f954ad9d640 1 -- 172.21.15.50:0/378156963 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c1409a0 con 0x7f954c13fe00 2024-04-02T16:22:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d4073380 con 0x7f05d4160470 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f05c8001670 con 0x7f05d4160470 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 >> v1:172.21.15.188:6789/0 conn(0x7f05d4160470 legacy=0x7f05d40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc0039b0 0x7f05bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc004800 0x7f05bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0051c0 0x7f05bc0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05d415cfd0 con 0x7f05bc004800 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc0148d0 con 0x7f05bc0039b0 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f05bc014ad0 con 0x7f05bc0051c0 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6792ffd640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f677c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f67937fe640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f677c005210 0x7f677c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f67927fc640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f677c004800 0x7f677c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7fa88bfff640 1 --2- 172.21.15.50:0/1154552177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c15b960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa8800097a0 tx=0x7fa88002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a3da858630d77b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f18d77fe640 1 -- 172.21.15.50:0/2367346062 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0317c0 con 0x7f18d8166c10 2024-04-02T16:22:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f18d77fe640 1 -- 172.21.15.50:0/2367346062 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18cc037070 con 0x7f18d8166c10 2024-04-02T16:22:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6792ffd640 1 -- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f677c004800 msgr2=0x7f677c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6792ffd640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f677c004800 0x7f677c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6792ffd640 1 -- 172.21.15.50:0/3216717582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f677c005210 msgr2=0x7f677c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f6792ffd640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f677c005210 0x7f677c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7fa88b7fe640 1 -- 172.21.15.50:0/1154552177 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880035020 con 0x7fa88c0d0930 2024-04-02T16:22:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f6792ffd640 1 -- 172.21.15.50:0/3216717582 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679415d110 con 0x7f677c0039d0 2024-04-02T16:22:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.063+0000 7f954ad9d640 1 --2- 172.21.15.50:0/378156963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c13fe00 0x7f954c13d620 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f954c067e10 tx=0x7f953c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f65f5f3a0571185 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febc59cd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7febb8001530 0x7febb8055420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febc59cd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febb8055960 0x7febb80077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f9549d9b640 1 -- 172.21.15.50:0/378156963 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c035020 con 0x7f954c13fe00 2024-04-02T16:22:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f9549d9b640 1 -- 172.21.15.50:0/378156963 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c031890 con 0x7f954c13fe00 2024-04-02T16:22:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febc59cd640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb8055dc0 con 0x7febb80df290 2024-04-02T16:22:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f9549d9b640 1 -- 172.21.15.50:0/378156963 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c031c60 con 0x7f954c13fe00 2024-04-02T16:22:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7fe01f59e640 1 --2- 172.21.15.50:0/2334920683 >> v2:172.21.15.50:3300/0 conn(0x7fe0180061a0 0x7fe018006630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f18d77fe640 1 -- 172.21.15.50:0/2367346062 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc031bc0 con 0x7f18d8166c10 2024-04-02T16:22:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febc59cd640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7febb8051c20 con 0x7febb8001530 2024-04-02T16:22:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febc59cd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7febb8051e50 con 0x7febb8055960 2024-04-02T16:22:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febbffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febb8055960 0x7febb80077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febbffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febb8055960 0x7febb80077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42460/0 (socket says 172.21.15.50:42460) 2024-04-02T16:22:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7febbffff640 1 -- 172.21.15.50:0/1050305723 learned_addr learned my addr 172.21.15.50:0/1050305723 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7fa88b7fe640 1 -- 172.21.15.50:0/1154552177 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa880031810 con 0x7fa88c0d0930 2024-04-02T16:22:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7fa88b7fe640 1 -- 172.21.15.50:0/1154552177 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880035020 con 0x7fa88c0d0930 2024-04-02T16:22:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7fa892ea0640 1 -- 172.21.15.50:0/1154552177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 msgr2=0x7fa88c15b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7fa892ea0640 1 --2- 172.21.15.50:0/1154552177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c15b960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa8800097a0 tx=0x7fa88002b8b0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 -- 172.21.15.50:0/2367346062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 msgr2=0x7f18d80af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7febbffff640 1 -- 172.21.15.50:0/1050305723 >> v2:172.21.15.50:3300/0 conn(0x7febb8001530 msgr2=0x7febb8055420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:22:26.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 --2- 172.21.15.50:0/2367346062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 0x7f18d80af580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18d806ef60 tx=0x7f18cc02b860 comp rx=0 tx=0).stop 2024-04-02T16:22:26.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d37fe640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc0039b0 0x7f05bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f3b69a2a640 1 --2- 172.21.15.50:0/2178942378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3d00 0x7f3b6406d230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b6415d3c0 tx=0x7f3b5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9e42e2ebfce205 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d3fff640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0051c0 0x7f05bc0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d2ffd640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc004800 0x7f05bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d37fe640 1 -- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0051c0 msgr2=0x7f05bc0141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d37fe640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0051c0 0x7f05bc0141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d37fe640 1 -- 172.21.15.50:0/3858034566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc004800 msgr2=0x7f05bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d37fe640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc004800 0x7f05bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d37fe640 1 -- 172.21.15.50:0/3858034566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d4073380 con 0x7f05bc0039b0 2024-04-02T16:22:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.064+0000 7f05d3fff640 1 -- 172.21.15.50:0/3858034566 reap_dead start 2024-04-02T16:22:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7fa892ea0640 1 -- 172.21.15.50:0/1154552177 shutdown_connections 2024-04-02T16:22:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7fa892ea0640 1 --2- 172.21.15.50:0/1154552177 >> v2:172.21.15.50:3300/0 conn(0x7fa88c0b31d0 0x7fa88c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7fa892ea0640 1 --2- 172.21.15.50:0/1154552177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c15b960 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7fa892ea0640 1 -- 172.21.15.50:0/1154552177 >> 172.21.15.50:0/1154552177 conn(0x7fa88c06b970 msgr2=0x7fa88c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7fa892ea0640 1 -- 172.21.15.50:0/1154552177 shutdown_connections 2024-04-02T16:22:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7febbffff640 1 --2- 172.21.15.50:0/1050305723 >> v2:172.21.15.50:3300/0 conn(0x7febb8001530 0x7febb8055420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7febbffff640 1 -- 172.21.15.50:0/1050305723 >> v1:172.21.15.50:6789/0 conn(0x7febb80df290 legacy=0x7febb8000e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7febbffff640 1 -- 172.21.15.50:0/1050305723 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb8052130 con 0x7febb8055960 2024-04-02T16:22:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7fa892ea0640 1 -- 172.21.15.50:0/1154552177 wait complete. 2024-04-02T16:22:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f6792ffd640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f677c003e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6784009860 tx=0x7f677c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e519ea1efc4f4719 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f3b69229640 1 -- 172.21.15.50:0/2178942378 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54035020 con 0x7f3b640b3d00 2024-04-02T16:22:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f3b69229640 1 -- 172.21.15.50:0/2178942378 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b54031640 con 0x7f3b640b3d00 2024-04-02T16:22:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f3b69229640 1 -- 172.21.15.50:0/2178942378 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5402d040 con 0x7f3b640b3d00 2024-04-02T16:22:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 -- 172.21.15.50:0/2367346062 shutdown_connections 2024-04-02T16:22:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 --2- 172.21.15.50:0/2367346062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 0x7f18d80af580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 --2- 172.21.15.50:0/2367346062 >> v2:172.21.15.50:3300/0 conn(0x7f18d80c55d0 0x7f18d80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 -- 172.21.15.50:0/2367346062 >> 172.21.15.50:0/2367346062 conn(0x7f18d80d1ef0 msgr2=0x7f18d80d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 -- 172.21.15.50:0/378156963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c13fe00 msgr2=0x7f954c13d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 --2- 172.21.15.50:0/378156963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c13fe00 0x7f954c13d620 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f954c067e10 tx=0x7f953c02b930 comp rx=0 tx=0).stop 2024-04-02T16:22:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7fe01ed9d640 1 --2- 172.21.15.50:0/2334920683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180e7820 0x7fe018038280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0100097a0 tx=0x7fe01002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be647df3f5831504 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 -- 172.21.15.50:0/2367346062 shutdown_connections 2024-04-02T16:22:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f18ddcaf640 1 -- 172.21.15.50:0/2367346062 wait complete. 2024-04-02T16:22:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784037070 con 0x7f677c0039d0 2024-04-02T16:22:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 -- 172.21.15.50:0/378156963 shutdown_connections 2024-04-02T16:22:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 --2- 172.21.15.50:0/378156963 >> v2:172.21.15.50:3300/0 conn(0x7f954c13ddb0 0x7f954c071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 --2- 172.21.15.50:0/378156963 >> v2:172.21.15.188:3300/0 conn(0x7f954c0cfea0 0x7f954c0748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 --2- 172.21.15.50:0/378156963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c13fe00 0x7f954c13d620 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 -- 172.21.15.50:0/378156963 >> 172.21.15.50:0/378156963 conn(0x7f954c067230 msgr2=0x7f954c067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6784032600 con 0x7f677c0039d0 2024-04-02T16:22:26.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f05d37fe640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc0039b0 0x7f05bc003dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05c400ba30 tx=0x7f05bc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee69d0a7905fcd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.065+0000 7f95510b1640 1 -- 172.21.15.50:0/378156963 shutdown_connections 2024-04-02T16:22:26.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f95510b1640 1 -- 172.21.15.50:0/378156963 wait complete. 2024-04-02T16:22:26.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4036020 con 0x7f05bc0039b0 2024-04-02T16:22:26.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c4032630 con 0x7f05bc0039b0 2024-04-02T16:22:26.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7febbffff640 1 --2- 172.21.15.50:0/1050305723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8055960 0x7febb80077b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febc0074810 tx=0x7febc00a9400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d48993e7d69a408 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3d00 msgr2=0x7f3b6406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f3b6c48f640 1 --2- 172.21.15.50:0/2178942378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3d00 0x7f3b6406d230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b6415d3c0 tx=0x7f3b5402b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f6791ffb640 1 -- 172.21.15.50:0/3216717582 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784009460 con 0x7f677c0039d0 2024-04-02T16:22:26.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7febbdffb640 1 -- 172.21.15.50:0/1050305723 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00b7070 con 0x7febb8055960 2024-04-02T16:22:26.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7febbdffb640 1 -- 172.21.15.50:0/1050305723 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc00af540 con 0x7febb8055960 2024-04-02T16:22:26.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7febbdffb640 1 -- 172.21.15.50:0/1050305723 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00af9b0 con 0x7febb8055960 2024-04-02T16:22:26.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f67994dd640 1 -- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 msgr2=0x7f677c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f67994dd640 1 --2- 172.21.15.50:0/3216717582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f677c003e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6784009860 tx=0x7f677c005830 comp rx=0 tx=0).stop 2024-04-02T16:22:26.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f05d27fc640 1 -- 172.21.15.50:0/3858034566 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c400b600 con 0x7f05bc0039b0 2024-04-02T16:22:26.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7f67937fe640 1 -- 172.21.15.50:0/3216717582 reap_dead start 2024-04-02T16:22:26.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7fe01dd9b640 1 -- 172.21.15.50:0/2334920683 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010037070 con 0x7fe0180e7820 2024-04-02T16:22:26.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f67994dd640 1 -- 172.21.15.50:0/3216717582 shutdown_connections 2024-04-02T16:22:26.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f67994dd640 1 -- 172.21.15.50:0/3216717582 >> 172.21.15.50:0/3216717582 conn(0x7f6794072b40 msgr2=0x7f6794072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f05d9af2640 1 -- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc0039b0 msgr2=0x7f05bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f05d9af2640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc0039b0 0x7f05bc003dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05c400ba30 tx=0x7f05bc005830 comp rx=0 tx=0).stop 2024-04-02T16:22:26.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fa892ea0640 1 Processor -- start 2024-04-02T16:22:26.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f05d9af2640 1 -- 172.21.15.50:0/3858034566 shutdown_connections 2024-04-02T16:22:26.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f05d9af2640 1 --2- 172.21.15.50:0/3858034566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc0039b0 0x7f05bc003dd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f05d9af2640 1 -- 172.21.15.50:0/3858034566 >> 172.21.15.50:0/3858034566 conn(0x7f05d406eb90 msgr2=0x7f05d406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f67994dd640 1 -- 172.21.15.50:0/3216717582 shutdown_connections 2024-04-02T16:22:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fa892ea0640 1 -- start start 2024-04-02T16:22:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7fe01dd9b640 1 -- 172.21.15.50:0/2334920683 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe010030710 con 0x7fe0180e7820 2024-04-02T16:22:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.066+0000 7fe01dd9b640 1 -- 172.21.15.50:0/2334920683 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010030b20 con 0x7fe0180e7820 2024-04-02T16:22:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 -- 172.21.15.50:0/2334920683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180e7820 msgr2=0x7fe018038280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 --2- 172.21.15.50:0/2334920683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180e7820 0x7fe018038280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0100097a0 tx=0x7fe01002b790 comp rx=0 tx=0).stop 2024-04-02T16:22:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f05d9af2640 1 -- 172.21.15.50:0/3858034566 shutdown_connections 2024-04-02T16:22:26.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f05d9af2640 1 -- 172.21.15.50:0/3858034566 wait complete. 2024-04-02T16:22:26.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f18ddcaf640 1 Processor -- start 2024-04-02T16:22:26.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 -- 172.21.15.50:0/2334920683 shutdown_connections 2024-04-02T16:22:26.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 --2- 172.21.15.50:0/2334920683 >> v2:172.21.15.50:3300/0 conn(0x7fe0180061a0 0x7fe018006630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 --2- 172.21.15.50:0/2334920683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180e7820 0x7fe018038280 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7febc59cd640 1 -- 172.21.15.50:0/1050305723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8055960 msgr2=0x7febb80077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f67994dd640 1 -- 172.21.15.50:0/3216717582 wait complete. 2024-04-02T16:22:26.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7febc59cd640 1 --2- 172.21.15.50:0/1050305723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8055960 0x7febb80077b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febc0074810 tx=0x7febc00a9400 comp rx=0 tx=0).stop 2024-04-02T16:22:26.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 -- 172.21.15.50:0/2334920683 >> 172.21.15.50:0/2334920683 conn(0x7fe0180559e0 msgr2=0x7fe018055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 shutdown_connections 2024-04-02T16:22:26.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f3b6c48f640 1 --2- 172.21.15.50:0/2178942378 >> v2:172.21.15.188:3300/0 conn(0x7f3b64166fe0 0x7f3b6406d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f3b6c48f640 1 --2- 172.21.15.50:0/2178942378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3d00 0x7f3b6406d230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 >> 172.21.15.50:0/2178942378 conn(0x7f3b6406eb90 msgr2=0x7f3b6406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 -- 172.21.15.50:0/2334920683 shutdown_connections 2024-04-02T16:22:26.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 shutdown_connections 2024-04-02T16:22:26.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7f95510b1640 1 Processor -- start 2024-04-02T16:22:26.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7febc59cd640 1 -- 172.21.15.50:0/1050305723 shutdown_connections 2024-04-02T16:22:26.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7febc59cd640 1 --2- 172.21.15.50:0/1050305723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8055960 0x7febb80077b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7f95510b1640 1 -- start start 2024-04-02T16:22:26.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7febc59cd640 1 --2- 172.21.15.50:0/1050305723 >> v2:172.21.15.50:3300/0 conn(0x7febb8001530 0x7febb8055420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7febc59cd640 1 -- 172.21.15.50:0/1050305723 >> 172.21.15.50:0/1050305723 conn(0x7febb80563e0 msgr2=0x7febb80567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7fe024805640 1 -- 172.21.15.50:0/2334920683 wait complete. 2024-04-02T16:22:26.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7f3b6c48f640 1 -- 172.21.15.50:0/2178942378 wait complete. 2024-04-02T16:22:26.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7f95510b1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c0cfea0 0x7f954c1333a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7f95510b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f954c13ddb0 0x7f954c138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f95510b1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f954c13fe00 0x7f954c1fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f95510b1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0d0480 con 0x7f954c13fe00 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f95510b1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0d0300 con 0x7f954c13ddb0 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f95510b1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f954c074e10 con 0x7f954c0cfea0 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa892ea0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88c0b31d0 0x7fa88c14b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.067+0000 7f18ddcaf640 1 -- start start 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.068+0000 7f18ddcaf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18d80c55d0 0x7f18d815d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18ddcaf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18d8162840 0x7f18d80c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18ddcaf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 0x7f18d80c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18ddcaf640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f18d806cea0 con 0x7f18d8162840 2024-04-02T16:22:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18ddcaf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f18d806cd20 con 0x7f18d8166c10 2024-04-02T16:22:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18ddcaf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f18d80afac0 con 0x7f18d80c55d0 2024-04-02T16:22:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f954a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f954c13ddb0 0x7f954c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa892ea0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa892ea0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c15d7c0 0x7fa88c0ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f67994dd640 1 Processor -- start 2024-04-02T16:22:26.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa892ea0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c0b3ba0 con 0x7fa88c0b31d0 2024-04-02T16:22:26.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa892ea0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c072520 con 0x7fa88c15d7c0 2024-04-02T16:22:26.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa892ea0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa88c06f210 con 0x7fa88c0d0930 2024-04-02T16:22:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f954ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c0cfea0 0x7f954c1333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f954b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f954c13fe00 0x7f954c1fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f954b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f954c13fe00 0x7f954c1fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45004/0 (socket says 172.21.15.50:45004) 2024-04-02T16:22:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f954ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c0cfea0 0x7f954c1333a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57116/0 (socket says 172.21.15.50:57116) 2024-04-02T16:22:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f954b59e640 1 -- 172.21.15.50:0/3598495086 learned_addr learned my addr 172.21.15.50:0/3598495086 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18d7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18d8162840 0x7f18d80c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa890c3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88c0b31d0 0x7fa88c14b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa890c3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88c0b31d0 0x7fa88c14b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45014/0 (socket says 172.21.15.50:45014) 2024-04-02T16:22:26.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa890c3c640 1 -- 172.21.15.50:0/819057760 learned_addr learned my addr 172.21.15.50:0/819057760 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7fa89143d640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c15d7c0 0x7fa88c0ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18d7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18d8162840 0x7f18d80c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45002/0 (socket says 172.21.15.50:45002) 2024-04-02T16:22:26.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18d7fff640 1 -- 172.21.15.50:0/3512028803 learned_addr learned my addr 172.21.15.50:0/3512028803 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18dd4ae640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 0x7f18d80c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f18dccad640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18d80c55d0 0x7f18d815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7fa88bfff640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f3b6c48f640 1 Processor -- start 2024-04-02T16:22:26.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f05d9af2640 1 Processor -- start 2024-04-02T16:22:26.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f05d9af2640 1 -- start start 2024-04-02T16:22:26.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7fa89143d640 1 -- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 msgr2=0x7fa88c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7fa89143d640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88c0d0930 0x7fa88c09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7fa89143d640 1 -- 172.21.15.50:0/819057760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88c0b31d0 msgr2=0x7fa88c14b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7fa89143d640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa88c0b31d0 0x7fa88c14b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f18d7fff640 1 -- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18d80c55d0 msgr2=0x7f18d815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f18d7fff640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18d80c55d0 0x7f18d815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f18d7fff640 1 -- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 msgr2=0x7f18d80c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f18d7fff640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18d8166c10 0x7f18d80c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7fa89143d640 1 -- 172.21.15.50:0/819057760 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c0cd840 con 0x7fa88c15d7c0 2024-04-02T16:22:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f18d7fff640 1 -- 172.21.15.50:0/3512028803 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d8164820 con 0x7f18d8162840 2024-04-02T16:22:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.069+0000 7f67994dd640 1 -- start start 2024-04-02T16:22:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67994dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f67940c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67994dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67940c6550 0x7f67940c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67994dd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67940c9ba0 0x7f67940b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67994dd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f679415cda0 con 0x7f67940c9ba0 2024-04-02T16:22:26.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67994dd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f679415d6d0 con 0x7f677c0039d0 2024-04-02T16:22:26.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67994dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f67940cc9b0 con 0x7f67940c6550 2024-04-02T16:22:26.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7fa89143d640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c15d7c0 0x7fa88c0ce8e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa884004980 tx=0x7fa8840105d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f05d9af2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc0039b0 0x7f05d40722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f05d9af2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d4201da0 0x7f05d41fae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f05d37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc0039b0 0x7f05d40722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f05d37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc0039b0 0x7f05d40722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45028/0 (socket says 172.21.15.50:45028) 2024-04-02T16:22:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88400be60 con 0x7fa88c15d7c0 2024-04-02T16:22:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67937fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67940c9ba0 0x7f67940b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f67937fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67940c9ba0 0x7f67940b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45034/0 (socket says 172.21.15.50:45034) 2024-04-02T16:22:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f6792ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f67940c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f67937fe640 1 -- 172.21.15.50:0/880276241 learned_addr learned my addr 172.21.15.50:0/880276241 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f05d9af2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d41585c0 0x7f05d4158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f05d37fe640 1 -- 172.21.15.50:0/4231287191 learned_addr learned my addr 172.21.15.50:0/4231287191 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f67927fc640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67940c6550 0x7f67940c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05d415fd60 con 0x7f05bc0039b0 2024-04-02T16:22:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f05d415cc60 con 0x7f05d4201da0 2024-04-02T16:22:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f05d415fee0 con 0x7f05d41585c0 2024-04-02T16:22:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f05d2ffd640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d4201da0 0x7f05d41fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f18d7fff640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18d8162840 0x7f18d80c59f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18d000cf60 tx=0x7f18d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f3b6c48f640 1 -- start start 2024-04-02T16:22:26.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f954ad9d640 1 -- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f954c13ddb0 msgr2=0x7f954c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f954ad9d640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f954c13ddb0 0x7f954c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f954ad9d640 1 -- 172.21.15.50:0/3598495086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f954c13fe00 msgr2=0x7f954c1fbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f954ad9d640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f954c13fe00 0x7f954c1fbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7f954ad9d640 1 -- 172.21.15.50:0/3598495086 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c13ac10 con 0x7f954c0cfea0 2024-04-02T16:22:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7fe024805640 1 Processor -- start 2024-04-02T16:22:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88401c070 con 0x7fa88c15d7c0 2024-04-02T16:22:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f6792ffd640 1 -- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67940c6550 msgr2=0x7f67940c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7fe024805640 1 -- start start 2024-04-02T16:22:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f6792ffd640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67940c6550 0x7f67940c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f6792ffd640 1 -- 172.21.15.50:0/880276241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67940c9ba0 msgr2=0x7f67940b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f6792ffd640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67940c9ba0 0x7f67940b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884017830 con 0x7fa88c15d7c0 2024-04-02T16:22:26.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7fa892ea0640 1 -- 172.21.15.50:0/819057760 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c0cdb20 con 0x7fa88c15d7c0 2024-04-02T16:22:26.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7fa892ea0640 1 -- 172.21.15.50:0/819057760 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c0ce0e0 con 0x7fa88c15d7c0 2024-04-02T16:22:26.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f6792ffd640 1 -- 172.21.15.50:0/880276241 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67940c7480 con 0x7f677c0039d0 2024-04-02T16:22:26.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f954ad9d640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c0cfea0 0x7f954c1333a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f954c0c5920 tx=0x7f953c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6c48f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3920 0x7f3b640cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6c48f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b640b3d00 0x7f3b640ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6c48f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b64166fe0 0x7f3b640aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6c48f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64162ef0 con 0x7f3b640b3920 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6c48f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64162d70 con 0x7f3b640b3d00 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6c48f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3b64163070 con 0x7f3b64166fe0 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6a22b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3920 0x7f3b640cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6a22b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3920 0x7f3b640cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45040/0 (socket says 172.21.15.50:45040) 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6a22b640 1 -- 172.21.15.50:0/3530940000 learned_addr learned my addr 172.21.15.50:0/3530940000 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b69a2a640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b640b3d00 0x7f3b640ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f6792ffd640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f67940c6010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c160 tx=0x7f67840047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c040070 con 0x7f954c0cfea0 2024-04-02T16:22:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c030ca0 con 0x7f954c0cfea0 2024-04-02T16:22:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c035040 con 0x7f954c0cfea0 2024-04-02T16:22:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f95510b1640 1 -- 172.21.15.50:0/3598495086 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c13aef0 con 0x7f954c0cfea0 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f95510b1640 1 -- 172.21.15.50:0/3598495086 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c072c50 con 0x7f954c0cfea0 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.071+0000 7f3b6aa2c640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b64166fe0 0x7f3b640aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f3b6a22b640 1 -- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b64166fe0 msgr2=0x7f3b640aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f3b6a22b640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b64166fe0 0x7f3b640aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f3b6a22b640 1 -- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b640b3d00 msgr2=0x7f3b640ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f3b6a22b640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b640b3d00 0x7f3b640ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f3b6a22b640 1 -- 172.21.15.50:0/3530940000 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b640aa160 con 0x7f3b640b3920 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784042020 con 0x7f677c0039d0 2024-04-02T16:22:26.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6784009440 con 0x7f677c0039d0 2024-04-02T16:22:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7fe024805640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0180061a0 0x7fe0180515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7fe024805640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0180e7820 0x7fe01804f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7fe024805640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180eb300 0x7fe018052670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7fe024805640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe0180ebe90 con 0x7fe0180e7820 2024-04-02T16:22:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7fe024805640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0180ec1a0 con 0x7fe0180061a0 2024-04-02T16:22:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7fe024805640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe018058ce0 con 0x7fe0180eb300 2024-04-02T16:22:26.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f3b6a22b640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3920 0x7f3b640cb9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000c4a0 tx=0x7f3b6000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f67994dd640 1 -- 172.21.15.50:0/880276241 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67940c7720 con 0x7f677c0039d0 2024-04-02T16:22:26.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f67994dd640 1 -- 172.21.15.50:0/880276241 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6794168d80 con 0x7f677c0039d0 2024-04-02T16:22:26.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7fe01f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180eb300 0x7fe018052670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7fe01f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180eb300 0x7fe018052670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57170/0 (socket says 172.21.15.50:57170) 2024-04-02T16:22:26.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7fe01f59e640 1 -- 172.21.15.50:0/4199706541 learned_addr learned my addr 172.21.15.50:0/4199706541 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f954c0756f0 con 0x7f954c0cfea0 2024-04-02T16:22:26.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f05d3fff640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d41585c0 0x7f05d4158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784037030 con 0x7f677c0039d0 2024-04-02T16:22:26.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6794075760 con 0x7f677c0039d0 2024-04-02T16:22:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa884014030 con 0x7fa88c15d7c0 2024-04-02T16:22:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60017070 con 0x7f3b640b3920 2024-04-02T16:22:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f05d37fe640 1 -- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d41585c0 msgr2=0x7f05d4158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f05d37fe640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05d41585c0 0x7f05d4158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f05d37fe640 1 -- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d4201da0 msgr2=0x7f05d41fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f05d37fe640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05d4201da0 0x7f05d41fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f05d37fe640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d4157e60 con 0x7f05bc0039b0 2024-04-02T16:22:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7febc59cd640 1 -- 172.21.15.50:0/1050305723 shutdown_connections 2024-04-02T16:22:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.070+0000 7febc59cd640 1 -- 172.21.15.50:0/1050305723 wait complete. 2024-04-02T16:22:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7febc59cd640 1 Processor -- start 2024-04-02T16:22:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7febc59cd640 1 -- start start 2024-04-02T16:22:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7febc59cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8001530 0x7febb80d9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fe01f59e640 1 -- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0180061a0 msgr2=0x7fe0180515b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T16:22:26.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fe01f59e640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0180061a0 0x7fe0180515b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fe01f59e640 1 -- 172.21.15.50:0/4199706541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0180e7820 msgr2=0x7fe01804f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fe01f59e640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0180e7820 0x7fe01804f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d001a070 con 0x7f18d8162840 2024-04-02T16:22:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d80c6560 con 0x7f18d8162840 2024-04-02T16:22:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fe01f59e640 1 -- 172.21.15.50:0/4199706541 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe018051af0 con 0x7fe0180eb300 2024-04-02T16:22:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d80c6b20 con 0x7f18d8162840 2024-04-02T16:22:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.072+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d000d840 con 0x7f18d8162840 2024-04-02T16:22:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d000dc10 con 0x7f18d8162840 2024-04-02T16:22:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18b8001030 con 0x7f18d8162840 2024-04-02T16:22:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fa892ea0640 1 -- 172.21.15.50:0/819057760 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa88c0756f0 con 0x7fa88c15d7c0 2024-04-02T16:22:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f05d37fe640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc0039b0 0x7f05d40722a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f05c402bf80 tx=0x7f05c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b60005ce0 con 0x7f3b640b3920 2024-04-02T16:22:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.073+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60007270 con 0x7f3b640b3920 2024-04-02T16:22:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b640aa440 con 0x7f3b640b3920 2024-04-02T16:22:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b640aaa00 con 0x7f3b640b3920 2024-04-02T16:22:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fa8897fa640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa87006c9c0 0x7fa87006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7febbeffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8001530 0x7febb80d9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7febbeffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8001530 0x7febb80d9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42524/0 (socket says 172.21.15.50:42524) 2024-04-02T16:22:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7febbeffd640 1 -- 172.21.15.50:0/875814988 learned_addr learned my addr 172.21.15.50:0/875814988 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7febc59cd640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8055960 0x7febb804a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7febc59cd640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febb80df290 0x7febb80e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.074+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa88408c2a0 con 0x7fa88c15d7c0 2024-04-02T16:22:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7fa8897fa640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6800/2935292990,v1:172.21.15.188:6801/2935292990] conn(0x7fa870076140 0x7fa870078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7febc59cd640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7febb8052c10 con 0x7febb80df290 2024-04-02T16:22:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7febc59cd640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7febb80526f0 con 0x7febb8001530 2024-04-02T16:22:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7febc59cd640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7febb80df670 con 0x7febb8055960 2024-04-02T16:22:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7febbe7fc640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8055960 0x7febb804a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 --> [v2:172.21.15.188:6800/2935292990,v1:172.21.15.188:6801/2935292990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa870078c50 con 0x7fa870076140 2024-04-02T16:22:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7fa88bfff640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6800/2935292990,v1:172.21.15.188:6801/2935292990] conn(0x7fa870076140 0x7fa870078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa884093050 con 0x7fa88c15d7c0 2024-04-02T16:22:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7febbffff640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febb80df290 0x7febb80e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4041070 con 0x7f05bc0039b0 2024-04-02T16:22:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c4030a40 con 0x7f05bc0039b0 2024-04-02T16:22:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4036040 con 0x7f05bc0039b0 2024-04-02T16:22:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7fe01f59e640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180eb300 0x7fe018052670 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe01400e880 tx=0x7fe01400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d4156dc0 con 0x7f05bc0039b0 2024-04-02T16:22:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d4157380 con 0x7f05bc0039b0 2024-04-02T16:22:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b64075760 con 0x7f3b640b3920 2024-04-02T16:22:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7fa88bfff640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6800/2935292990,v1:172.21.15.188:6801/2935292990] conn(0x7fa870076140 0x7fa870078580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7fa890c3c640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa87006c9c0 0x7fa87006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7febbffff640 1 -- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8055960 msgr2=0x7febb804a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7febbffff640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8055960 0x7febb804a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7febbffff640 1 -- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8001530 msgr2=0x7febb80d9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7febbffff640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8001530 0x7febb80d9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7febbffff640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb80dc160 con 0x7febb80df290 2024-04-02T16:22:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014017070 con 0x7fe0180eb300 2024-04-02T16:22:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe014004020 con 0x7fe0180eb300 2024-04-02T16:22:26.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7febbffff640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febb80df290 0x7febb80e1340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febc0073ed0 tx=0x7febc006b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140043f0 con 0x7fe0180eb300 2024-04-02T16:22:26.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b6001a070 con 0x7f3b640b3920 2024-04-02T16:22:26.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7febbeffd640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb8001530 0x7febb80d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6784045020 con 0x7f677c0039d0 2024-04-02T16:22:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f677b7fe640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f678006c970 0x7f678006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.076+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f67840ad900 con 0x7f677c0039d0 2024-04-02T16:22:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f677b7fe640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6808/3730321182,v1:172.21.15.188:6809/3730321182] conn(0x7f67800760e0 0x7f6780078520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 --> [v2:172.21.15.188:6808/3730321182,v1:172.21.15.188:6809/3730321182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f678403ae90 con 0x7f67800760e0 2024-04-02T16:22:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67840adde0 con 0x7f677c0039d0 2024-04-02T16:22:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00b2910 con 0x7febb80df290 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc00b2ab0 con 0x7febb80df290 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18d001d070 con 0x7f18d8162840 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f18d57fa640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f18c406c9c0 0x7f18c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.075+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f953c04b060 con 0x7f954c0cfea0 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f95337fe640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f953806c9f0 0x7f953806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f953c0add20 con 0x7f954c0cfea0 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f95337fe640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.50:6800/2132969240,v1:172.21.15.50:6801/2132969240] conn(0x7f9538076160 0x7f95380785a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 --> [v2:172.21.15.50:6800/2132969240,v1:172.21.15.50:6801/2132969240] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9538078c70 con 0x7f9538076160 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f953c0ae110 con 0x7f954c0cfea0 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f3b5affd640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3b4806c940 0x7f3b4806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3b6008b0b0 con 0x7f3b640b3920 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f3b5affd640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6824/1410600395,v1:172.21.15.188:6825/1410600395] conn(0x7f3b48076040 0x7f3b48078480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 --> [v2:172.21.15.188:6824/1410600395,v1:172.21.15.188:6825/1410600395] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b60020e50 con 0x7f3b48076040 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b60092050 con 0x7f3b640b3920 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f3b6aa2c640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6824/1410600395,v1:172.21.15.188:6825/1410600395] conn(0x7f3b48076040 0x7f3b48078480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05d4075760 con 0x7f05bc0039b0 2024-04-02T16:22:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f67937fe640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6808/3730321182,v1:172.21.15.188:6809/3730321182] conn(0x7f67800760e0 0x7f6780078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f67927fc640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f678006c970 0x7f678006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7f67937fe640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6808/3730321182,v1:172.21.15.188:6809/3730321182] conn(0x7f67800760e0 0x7f6780078520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== osd.7 v2:172.21.15.188:6800/2935292990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa870078c50 con 0x7fa870076140 2024-04-02T16:22:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f18d008d1a0 con 0x7f18d8162840 2024-04-02T16:22:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f18dccad640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f18c406c9c0 0x7f18c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f954a59c640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f953806c9f0 0x7f953806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f954b59e640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.50:6800/2132969240,v1:172.21.15.50:6801/2132969240] conn(0x7f9538076160 0x7f95380785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f3b69a2a640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3b4806c940 0x7f3b4806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7febc59cd640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb80dc440 con 0x7febb80df290 2024-04-02T16:22:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7febc59cd640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb80e0e20 con 0x7febb80df290 2024-04-02T16:22:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f18d57fa640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:6824/815998529,v1:172.21.15.50:6825/815998529] conn(0x7f18c4076140 0x7f18c4078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 --> [v2:172.21.15.50:6824/815998529,v1:172.21.15.50:6825/815998529] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18d000d840 con 0x7f18c4076140 2024-04-02T16:22:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f18d008d590 con 0x7f18d8162840 2024-04-02T16:22:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe018050510 con 0x7fe0180eb300 2024-04-02T16:22:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.077+0000 7fa890c3c640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa87006c9c0 0x7fa87006ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa878007fd0 tx=0x7fa878009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f67927fc640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f678006c970 0x7f678006ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f67940c6c70 tx=0x7f6788001560 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe018050ad0 con 0x7fe0180eb300 2024-04-02T16:22:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f18dccad640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f18c406c9c0 0x7f18c406ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18d80cbc50 tx=0x7f18c8007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f18dd4ae640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:6824/815998529,v1:172.21.15.50:6825/815998529] conn(0x7f18c4076140 0x7f18c4078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f954b59e640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.50:6800/2132969240,v1:172.21.15.50:6801/2132969240] conn(0x7f9538076160 0x7f95380785a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f954a59c640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f953806c9f0 0x7f953806eeb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f954c1379d0 tx=0x7f9540008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f3b69a2a640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3b4806c940 0x7f3b4806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b54036000 tx=0x7f3b5403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05c404c060 con 0x7f05bc0039b0 2024-04-02T16:22:26.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== osd.6 v2:172.21.15.188:6808/3730321182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f678403ae90 con 0x7f67800760e0 2024-04-02T16:22:26.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7f18dd4ae640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:6824/815998529,v1:172.21.15.50:6825/815998529] conn(0x7f18c4076140 0x7f18c4078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7f05bbfff640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f05c006c9c0 0x7f05c006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f05c40ada00 con 0x7f05bc0039b0 2024-04-02T16:22:26.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7f05d2ffd640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f05c006c9c0 0x7f05c006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7f05bbfff640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6816/924583379,v1:172.21.15.188:6817/924583379] conn(0x7f05c00760c0 0x7f05c0078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.188:6816/924583379,v1:172.21.15.188:6817/924583379] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05c4032630 con 0x7f05c00760c0 2024-04-02T16:22:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.079+0000 7fe0057fa640 1 -- 172.21.15.50:0/4199706541 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdff8001030 con 0x7fe0180eb300 2024-04-02T16:22:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.078+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00b2e60 con 0x7febb80df290 2024-04-02T16:22:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.080+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== osd.3 v2:172.21.15.50:6800/2132969240 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9538078c70 con 0x7f9538076160 2024-04-02T16:22:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.080+0000 7f05d2ffd640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f05c006c9c0 0x7f05c006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05d41449a0 tx=0x7f05cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.080+0000 7febc59cd640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febb80dc680 con 0x7febb80df290 2024-04-02T16:22:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.080+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== osd.2 v2:172.21.15.50:6824/815998529 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18d000d840 con 0x7f18c4076140 2024-04-02T16:22:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.080+0000 7f3b6aa2c640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6824/1410600395,v1:172.21.15.188:6825/1410600395] conn(0x7f3b48076040 0x7f3b48078480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.080+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f05c40b4050 con 0x7f05bc0039b0 2024-04-02T16:22:26.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.080+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== osd.4 v2:172.21.15.188:6824/1410600395 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b60020e50 con 0x7f3b48076040 2024-04-02T16:22:26.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.081+0000 7f05d3fff640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6816/924583379,v1:172.21.15.188:6817/924583379] conn(0x7f05c00760c0 0x7f05c0078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.081+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe01400c940 con 0x7fe0180eb300 2024-04-02T16:22:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.081+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febc008b030 con 0x7febb80df290 2024-04-02T16:22:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.081+0000 7feba7fff640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7febac06c9f0 0x7febac06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.081+0000 7fe0077fe640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe00c06c9c0 0x7fe00c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.081+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe01408bc40 con 0x7fe0180eb300 2024-04-02T16:22:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.081+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7febc011a880 con 0x7febb80df290 2024-04-02T16:22:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7febbeffd640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7febac06c9f0 0x7febac06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7feba7fff640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:6808/3677185598,v1:172.21.15.50:6810/3677185598] conn(0x7febac076170 0x7febac0785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7fe0077fe640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.50:6809/3280203769,v1:172.21.15.50:6811/3280203769] conn(0x7fe00c076130 0x7fe00c078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7fe01ed9d640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe00c06c9c0 0x7fe00c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7febbe7fc640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:6808/3677185598,v1:172.21.15.50:6810/3677185598] conn(0x7febac076170 0x7febac0785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 --> [v2:172.21.15.50:6809/3280203769,v1:172.21.15.50:6811/3280203769] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe00c078c40 con 0x7fe00c076130 2024-04-02T16:22:26.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7febbeffd640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7febac06c9f0 0x7febac06eeb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febb0001780 tx=0x7febb0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe014092020 con 0x7fe0180eb300 2024-04-02T16:22:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.082+0000 7fe01e59c640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.50:6809/3280203769,v1:172.21.15.50:6811/3280203769] conn(0x7fe00c076130 0x7fe00c078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.083+0000 7fe01ed9d640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe00c06c9c0 0x7fe00c06ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe010034000 tx=0x7fe01002f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.083+0000 7fe01e59c640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.50:6809/3280203769,v1:172.21.15.50:6811/3280203769] conn(0x7fe00c076130 0x7fe00c078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.083+0000 7febbe7fc640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:6808/3677185598,v1:172.21.15.50:6810/3677185598] conn(0x7febac076170 0x7febac0785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.083+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.50:6808/3677185598,v1:172.21.15.50:6810/3677185598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febc007ee50 con 0x7febac076170 2024-04-02T16:22:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.083+0000 7f05d3fff640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6816/924583379,v1:172.21.15.188:6817/924583379] conn(0x7f05c00760c0 0x7f05c0078500 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.083+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7febc011f2a0 con 0x7febb80df290 2024-04-02T16:22:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.084+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== osd.1 v2:172.21.15.50:6809/3280203769 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe00c078c40 con 0x7fe00c076130 2024-04-02T16:22:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.084+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== osd.0 v2:172.21.15.50:6808/3677185598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febc007ee50 con 0x7febac076170 2024-04-02T16:22:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.085+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== osd.5 v2:172.21.15.188:6816/924583379 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05c4032630 con 0x7f05c00760c0 2024-04-02T16:22:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.111+0000 7fa892ea0640 1 -- 172.21.15.50:0/819057760 --> [v2:172.21.15.188:6800/2935292990,v1:172.21.15.188:6801/2935292990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa88c1551d0 con 0x7fa870076140 2024-04-02T16:22:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.111+0000 7fa8897fa640 1 -- 172.21.15.50:0/819057760 <== osd.7 v2:172.21.15.188:6800/2935292990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa88c1551d0 con 0x7fa870076140 2024-04-02T16:22:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.112+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 --> [v2:172.21.15.188:6808/3730321182,v1:172.21.15.188:6809/3730321182] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67940c6930 con 0x7f67800760e0 2024-04-02T16:22:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.112+0000 7fa876ffd640 1 -- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6800/2935292990,v1:172.21.15.188:6801/2935292990] conn(0x7fa870076140 msgr2=0x7fa870078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.112+0000 7fa876ffd640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6800/2935292990,v1:172.21.15.188:6801/2935292990] conn(0x7fa870076140 0x7fa870078580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.112+0000 7fa876ffd640 1 -- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa87006c9c0 msgr2=0x7fa87006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.112+0000 7fa876ffd640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa87006c9c0 0x7fa87006ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa878007fd0 tx=0x7fa878009040 comp rx=0 tx=0).stop 2024-04-02T16:22:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.112+0000 7fa876ffd640 1 -- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c15d7c0 msgr2=0x7fa88c0ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.112+0000 7fa876ffd640 1 --2- 172.21.15.50:0/819057760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88c15d7c0 0x7fa88c0ce8e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa884004980 tx=0x7fa8840105d0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 --> [v2:172.21.15.188:6824/1410600395,v1:172.21.15.188:6825/1410600395] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b640aae50 con 0x7f3b48076040 2024-04-02T16:22:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7fa89143d640 1 -- 172.21.15.50:0/819057760 reap_dead start 2024-04-02T16:22:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f677b7fe640 1 -- 172.21.15.50:0/880276241 <== osd.6 v2:172.21.15.188:6808/3730321182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f67940c6930 con 0x7f67800760e0 2024-04-02T16:22:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7fa876ffd640 1 -- 172.21.15.50:0/819057760 shutdown_connections 2024-04-02T16:22:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7fa876ffd640 1 -- 172.21.15.50:0/819057760 >> 172.21.15.50:0/819057760 conn(0x7fa88c06b970 msgr2=0x7fa88c06bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 --> [v2:172.21.15.50:6800/2132969240,v1:172.21.15.50:6801/2132969240] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f954c13e240 con 0x7f9538076160 2024-04-02T16:22:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7fa876ffd640 1 -- 172.21.15.50:0/819057760 shutdown_connections 2024-04-02T16:22:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7fa876ffd640 1 -- 172.21.15.50:0/819057760 wait complete. 2024-04-02T16:22:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6808/3730321182,v1:172.21.15.188:6809/3730321182] conn(0x7f67800760e0 msgr2=0x7f6780078520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6808/3730321182,v1:172.21.15.188:6809/3730321182] conn(0x7f67800760e0 0x7f6780078520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f3b5affd640 1 -- 172.21.15.50:0/3530940000 <== osd.4 v2:172.21.15.188:6824/1410600395 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3b640aae50 con 0x7f3b48076040 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f678006c970 msgr2=0x7f678006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f678006c970 0x7f678006ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f67940c6c70 tx=0x7f6788001560 comp rx=0 tx=0).stop 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 msgr2=0x7f67940c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 --2- 172.21.15.50:0/880276241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f677c0039d0 0x7f67940c6010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c160 tx=0x7f67840047d0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67937fe640 1 -- 172.21.15.50:0/880276241 reap_dead start 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 shutdown_connections 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 >> 172.21.15.50:0/880276241 conn(0x7f6794072b40 msgr2=0x7f6794159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f95337fe640 1 -- 172.21.15.50:0/3598495086 <== osd.3 v2:172.21.15.50:6800/2132969240 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f954c13e240 con 0x7f9538076160 2024-04-02T16:22:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.113+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 --> [v2:172.21.15.50:6824/815998529,v1:172.21.15.50:6825/815998529] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18b8002d10 con 0x7f18c4076140 2024-04-02T16:22:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f18d57fa640 1 -- 172.21.15.50:0/3512028803 <== osd.2 v2:172.21.15.50:6824/815998529 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f18b8002d10 con 0x7f18c4076140 2024-04-02T16:22:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 >> [v2:172.21.15.50:6800/2132969240,v1:172.21.15.50:6801/2132969240] conn(0x7f9538076160 msgr2=0x7f95380785a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6824/1410600395,v1:172.21.15.188:6825/1410600395] conn(0x7f3b48076040 msgr2=0x7f3b48078480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6824/1410600395,v1:172.21.15.188:6825/1410600395] conn(0x7f3b48076040 0x7f3b48078480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 shutdown_connections 2024-04-02T16:22:26.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.50:6800/2132969240,v1:172.21.15.50:6801/2132969240] conn(0x7f9538076160 0x7f95380785a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f953806c9f0 msgr2=0x7f953806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f953806c9f0 0x7f953806eeb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f954c1379d0 tx=0x7f9540008040 comp rx=0 tx=0).stop 2024-04-02T16:22:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c0cfea0 msgr2=0x7f954c1333a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 --2- 172.21.15.50:0/3598495086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954c0cfea0 0x7f954c1333a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f954c0c5920 tx=0x7f953c0046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f954b59e640 1 -- 172.21.15.50:0/3598495086 reap_dead start 2024-04-02T16:22:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3b4806c940 msgr2=0x7f3b4806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3b4806c940 0x7f3b4806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b54036000 tx=0x7f3b5403c040 comp rx=0 tx=0).stop 2024-04-02T16:22:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 shutdown_connections 2024-04-02T16:22:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 >> 172.21.15.50:0/3598495086 conn(0x7f954c067230 msgr2=0x7f954c0b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:6824/815998529,v1:172.21.15.50:6825/815998529] conn(0x7f18c4076140 msgr2=0x7f18c4078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3920 msgr2=0x7f3b640cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 --2- 172.21.15.50:0/3530940000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b640b3920 0x7f3b640cb9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000c4a0 tx=0x7f3b6000c960 comp rx=0 tx=0).stop 2024-04-02T16:22:26.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6aa2c640 1 -- 172.21.15.50:0/3530940000 reap_dead start 2024-04-02T16:22:26.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f18ddcaf640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:6824/815998529,v1:172.21.15.50:6825/815998529] conn(0x7f18c4076140 0x7f18c4078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 shutdown_connections 2024-04-02T16:22:26.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 >> 172.21.15.50:0/3530940000 conn(0x7f3b6406eb90 msgr2=0x7f3b6406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.114+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 shutdown_connections 2024-04-02T16:22:26.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f67797fa640 1 -- 172.21.15.50:0/880276241 wait complete. 2024-04-02T16:22:26.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f95317fa640 1 -- 172.21.15.50:0/3598495086 wait complete. 2024-04-02T16:22:26.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f18c406c9c0 msgr2=0x7f18c406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f18c406c9c0 0x7f18c406ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18d80cbc50 tx=0x7f18c8007e80 comp rx=0 tx=0).stop 2024-04-02T16:22:26.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18d8162840 msgr2=0x7f18d80c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 --2- 172.21.15.50:0/3512028803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18d8162840 0x7f18d80c59f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18d000cf60 tx=0x7f18d000cf90 comp rx=0 tx=0).stop 2024-04-02T16:22:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18dd4ae640 1 -- 172.21.15.50:0/3512028803 reap_dead start 2024-04-02T16:22:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 shutdown_connections 2024-04-02T16:22:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f3b6c48f640 1 -- 172.21.15.50:0/3530940000 wait complete. 2024-04-02T16:22:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 shutdown_connections 2024-04-02T16:22:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 >> 172.21.15.50:0/3512028803 conn(0x7f18d80d1ef0 msgr2=0x7f18d806f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 shutdown_connections 2024-04-02T16:22:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.115+0000 7f18ddcaf640 1 -- 172.21.15.50:0/3512028803 wait complete. 2024-04-02T16:22:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.116+0000 7fe0057fa640 1 -- 172.21.15.50:0/4199706541 --> [v2:172.21.15.50:6809/3280203769,v1:172.21.15.50:6811/3280203769] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdff8002dd0 con 0x7fe00c076130 2024-04-02T16:22:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.116+0000 7febc59cd640 1 -- 172.21.15.50:0/875814988 --> [v2:172.21.15.50:6808/3677185598,v1:172.21.15.50:6810/3677185598] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febb80e0980 con 0x7febac076170 2024-04-02T16:22:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.116+0000 7fe0077fe640 1 -- 172.21.15.50:0/4199706541 <== osd.1 v2:172.21.15.50:6809/3280203769 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdff8002dd0 con 0x7fe00c076130 2024-04-02T16:22:26.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.117+0000 7feba7fff640 1 -- 172.21.15.50:0/875814988 <== osd.0 v2:172.21.15.50:6808/3677185598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7febb4009530 con 0x7febac076170 2024-04-02T16:22:26.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.117+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 >> [v2:172.21.15.50:6809/3280203769,v1:172.21.15.50:6811/3280203769] conn(0x7fe00c076130 msgr2=0x7fe00c078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.117+0000 7fe024805640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.50:6809/3280203769,v1:172.21.15.50:6811/3280203769] conn(0x7fe00c076130 0x7fe00c078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.117+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe00c06c9c0 msgr2=0x7fe00c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.117+0000 7fe024805640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe00c06c9c0 0x7fe00c06ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe010034000 tx=0x7fe01002f000 comp rx=0 tx=0).stop 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.117+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180eb300 msgr2=0x7fe018052670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.118+0000 7fe024805640 1 --2- 172.21.15.50:0/4199706541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0180eb300 0x7fe018052670 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe01400e880 tx=0x7fe01400ed40 comp rx=0 tx=0).stop 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.118+0000 7fe01f59e640 1 -- 172.21.15.50:0/4199706541 reap_dead start 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.118+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 shutdown_connections 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.118+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 >> 172.21.15.50:0/4199706541 conn(0x7fe0180559e0 msgr2=0x7fe018039b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.118+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 shutdown_connections 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.118+0000 7fe024805640 1 -- 172.21.15.50:0/4199706541 wait complete. 2024-04-02T16:22:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.118+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 --> [v2:172.21.15.188:6816/924583379,v1:172.21.15.188:6817/924583379] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f05d40cd880 con 0x7f05c00760c0 2024-04-02T16:22:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7feba5ffb640 1 -- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:6808/3677185598,v1:172.21.15.50:6810/3677185598] conn(0x7febac076170 msgr2=0x7febac0785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7feba5ffb640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:6808/3677185598,v1:172.21.15.50:6810/3677185598] conn(0x7febac076170 0x7febac0785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7feba5ffb640 1 -- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7febac06c9f0 msgr2=0x7febac06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7feba5ffb640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7febac06c9f0 0x7febac06eeb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febb0001780 tx=0x7febb0009040 comp rx=0 tx=0).stop 2024-04-02T16:22:26.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7feba5ffb640 1 -- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febb80df290 msgr2=0x7febb80e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7feba5ffb640 1 --2- 172.21.15.50:0/875814988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febb80df290 0x7febb80e1340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febc0073ed0 tx=0x7febc006b3e0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7febbffff640 1 -- 172.21.15.50:0/875814988 reap_dead start 2024-04-02T16:22:26.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.119+0000 7f05bbfff640 1 -- 172.21.15.50:0/4231287191 <== osd.5 v2:172.21.15.188:6816/924583379 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f05d40cd880 con 0x7f05c00760c0 2024-04-02T16:22:26.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.120+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6816/924583379,v1:172.21.15.188:6817/924583379] conn(0x7f05c00760c0 msgr2=0x7f05c0078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.120+0000 7f05d9af2640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6816/924583379,v1:172.21.15.188:6817/924583379] conn(0x7f05c00760c0 0x7f05c0078500 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.120+0000 7feba5ffb640 1 -- 172.21.15.50:0/875814988 shutdown_connections 2024-04-02T16:22:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.120+0000 7feba5ffb640 1 -- 172.21.15.50:0/875814988 >> 172.21.15.50:0/875814988 conn(0x7febb80563e0 msgr2=0x7febb803b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f05c006c9c0 msgr2=0x7f05c006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f05c006c9c0 0x7f05c006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05d41449a0 tx=0x7f05cc009040 comp rx=0 tx=0).stop 2024-04-02T16:22:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc0039b0 msgr2=0x7f05d40722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 --2- 172.21.15.50:0/4231287191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05bc0039b0 0x7f05d40722a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f05c402bf80 tx=0x7f05c4004730 comp rx=0 tx=0).stop 2024-04-02T16:22:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d3fff640 1 -- 172.21.15.50:0/4231287191 reap_dead start 2024-04-02T16:22:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7feba5ffb640 1 -- 172.21.15.50:0/875814988 shutdown_connections 2024-04-02T16:22:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 shutdown_connections 2024-04-02T16:22:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 >> 172.21.15.50:0/4231287191 conn(0x7f05d406eb90 msgr2=0x7f05d40cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7feba5ffb640 1 -- 172.21.15.50:0/875814988 wait complete. 2024-04-02T16:22:26.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 shutdown_connections 2024-04-02T16:22:26.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.121+0000 7f05d9af2640 1 -- 172.21.15.50:0/4231287191 wait complete. 2024-04-02T16:22:26.183 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:26.184 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:26.184 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:26.185 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-02T16:22:26.186 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-02T16:22:26.186 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-02T16:22:26.187 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.187 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-02T16:22:26.188 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.188 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-02T16:22:26.189 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.189 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-02T16:22:26.190 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:26.190 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-02T16:22:26.191 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:26.191 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-02T16:22:26.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.398+0000 7f4212d60640 1 Processor -- start 2024-04-02T16:22:26.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.398+0000 7f60cffff640 1 Processor -- start 2024-04-02T16:22:26.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.398+0000 7f36893af640 1 Processor -- start 2024-04-02T16:22:26.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.398+0000 7f36893af640 1 -- start start 2024-04-02T16:22:26.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.398+0000 7f4212d60640 1 -- start start 2024-04-02T16:22:26.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7fb69e338640 1 Processor -- start 2024-04-02T16:22:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.398+0000 7f60cffff640 1 -- start start 2024-04-02T16:22:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7faad0315640 1 Processor -- start 2024-04-02T16:22:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f60cffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f60c80e7a40 0x7f60c8049b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f60cffff640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c80e7e20 con 0x7f60c803c350 2024-04-02T16:22:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f60cffff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c803b590 con 0x7f60c804a090 2024-04-02T16:22:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f60cffff640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f60c803b870 con 0x7f60c80e7a40 2024-04-02T16:22:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f36893af640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f3684166db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f36893af640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f36840b32b0 0x7f36840b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f36893af640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36841662f0 con 0x7f36840b5dc0 2024-04-02T16:22:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f36893af640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f36840d3a80 con 0x7f36840b32b0 2024-04-02T16:22:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f60cf7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f60c803c350 0x7f60c8051310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55018/0 (socket says 172.21.15.50:55018) 2024-04-02T16:22:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f60cf7fe640 1 -- 172.21.15.50:0/606234982 learned_addr learned my addr 172.21.15.50:0/606234982 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f36893af640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36840d3d60 con 0x7f36840b7df0 2024-04-02T16:22:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7f36827fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f36840b32b0 0x7f36840b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f36827fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f36840b32b0 0x7f36840b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45056/0 (socket says 172.21.15.50:45056) 2024-04-02T16:22:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f4212d60640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c0d0790 0x7f420c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f4212d60640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f420c06edb0 0x7f420c0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f4212d60640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f420c0b3730 con 0x7f420c06c010 2024-04-02T16:22:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f4212d60640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f420c15a770 con 0x7f420c06edb0 2024-04-02T16:22:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f4212d60640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0b4060 con 0x7f420c0d0790 2024-04-02T16:22:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f36827fc640 1 -- 172.21.15.50:0/4173408412 learned_addr learned my addr 172.21.15.50:0/4173408412 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f3682ffd640 1 --2- 172.21.15.50:0/4173408412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f3684166db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7faad0315640 1 -- start start 2024-04-02T16:22:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f60cdffb640 1 -- 172.21.15.50:0/606234982 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3879250761 0 0) 0x7f60c80e7e20 con 0x7f60c803c350 2024-04-02T16:22:26.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f42112fd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f420c06edb0 0x7f420c0acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f60cdffb640 1 -- 172.21.15.50:0/606234982 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60b8003520 con 0x7f60c803c350 2024-04-02T16:22:26.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f42112fd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f420c06edb0 0x7f420c0acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45068/0 (socket says 172.21.15.50:45068) 2024-04-02T16:22:26.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7f42112fd640 1 -- 172.21.15.50:0/3966579200 learned_addr learned my addr 172.21.15.50:0/3966579200 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.399+0000 7fb69e338640 1 -- start start 2024-04-02T16:22:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7fb69e338640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb698162660 0x7fb698072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7fb69e338640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb698162a40 con 0x7fb6980b4760 2024-04-02T16:22:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7fb69e338640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6980b2e30 con 0x7fb69806d1f0 2024-04-02T16:22:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.400+0000 7fb69e338640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb6980734d0 con 0x7fb698162660 2024-04-02T16:22:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f79a9d40640 1 Processor -- start 2024-04-02T16:22:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f79a9d40640 1 -- start start 2024-04-02T16:22:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7faad0315640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faac80afd80 0x7faac815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7faad0315640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faac806e820 0x7faac815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f3682ffd640 1 -- 172.21.15.50:0/4173408412 >> v2:172.21.15.50:3300/0 conn(0x7f36840b32b0 msgr2=0x7f36840b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f3682ffd640 1 --2- 172.21.15.50:0/4173408412 >> v2:172.21.15.50:3300/0 conn(0x7f36840b32b0 0x7f36840b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f3682ffd640 1 -- 172.21.15.50:0/4173408412 >> v1:172.21.15.188:6789/0 conn(0x7f36840b5dc0 legacy=0x7f36840d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f3682ffd640 1 -- 172.21.15.50:0/4173408412 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36840d4040 con 0x7f36840b7df0 2024-04-02T16:22:26.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f36827fc640 1 --2- 172.21.15.50:0/4173408412 >> v2:172.21.15.50:3300/0 conn(0x7f36840b32b0 0x7f36840b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:26.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f60ceffd640 1 --2- 172.21.15.50:0/606234982 >> v2:172.21.15.50:3300/0 conn(0x7f60c80e7a40 0x7f60c8049b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f60cdffb640 1 -- 172.21.15.50:0/606234982 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 727527739 0 0) 0x7f60c803b590 con 0x7f60c804a090 2024-04-02T16:22:26.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f60cdffb640 1 -- 172.21.15.50:0/606234982 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c80e7e20 con 0x7f60c804a090 2024-04-02T16:22:26.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7faad0315640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac815b780 con 0x7faac80748d0 2024-04-02T16:22:26.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7faad0315640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7faac815bd40 con 0x7faac80afd80 2024-04-02T16:22:26.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7faad0315640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7faac815ba60 con 0x7faac806e820 2024-04-02T16:22:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f42112fd640 1 -- 172.21.15.50:0/3966579200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c0d0790 msgr2=0x7f420c15a230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:22:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f42112fd640 1 --2- 172.21.15.50:0/3966579200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c0d0790 0x7f420c15a230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f42112fd640 1 -- 172.21.15.50:0/3966579200 >> v1:172.21.15.50:6789/0 conn(0x7f420c06c010 legacy=0x7f420c0d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f42112fd640 1 -- 172.21.15.50:0/3966579200 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c0b3a10 con 0x7f420c06edb0 2024-04-02T16:22:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f60ceffd640 1 -- 172.21.15.50:0/606234982 >> v1:172.21.15.188:6789/0 conn(0x7f60c804a090 legacy=0x7f60c8054890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f60ceffd640 1 -- 172.21.15.50:0/606234982 >> v1:172.21.15.50:6789/0 conn(0x7f60c803c350 legacy=0x7f60c8051310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7f60ceffd640 1 -- 172.21.15.50:0/606234982 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c803bb50 con 0x7f60c80e7a40 2024-04-02T16:22:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faacd8b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faac806e820 0x7faac815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7fb6977fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb6980b4760 0x7fb6980b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55034/0 (socket says 172.21.15.50:55034) 2024-04-02T16:22:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7fb6977fe640 1 -- 172.21.15.50:0/3530121379 learned_addr learned my addr 172.21.15.50:0/3530121379 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.401+0000 7fb697fff640 1 --2- 172.21.15.50:0/3530121379 >> v2:172.21.15.50:3300/0 conn(0x7fb698162660 0x7fb698072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7fb697fff640 1 -- 172.21.15.50:0/3530121379 >> v1:172.21.15.188:6789/0 conn(0x7fb69806d1f0 legacy=0x7fb6980acd70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:22:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7fb697fff640 1 -- 172.21.15.50:0/3530121379 >> v1:172.21.15.50:6789/0 conn(0x7fb6980b4760 legacy=0x7fb6980b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7fb697fff640 1 -- 172.21.15.50:0/3530121379 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6980b3110 con 0x7fb698162660 2024-04-02T16:22:26.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faacd8b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faac806e820 0x7faac815ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42548/0 (socket says 172.21.15.50:42548) 2024-04-02T16:22:26.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faacd8b0640 1 -- 172.21.15.50:0/3419431817 learned_addr learned my addr 172.21.15.50:0/3419431817 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7f3682ffd640 1 --2- 172.21.15.50:0/4173408412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f3684166db0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f36740097d0 tx=0x7f367402b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606d5142502fee3b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7f42112fd640 1 --2- 172.21.15.50:0/3966579200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06edb0 0x7f420c0acf20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009870 tx=0x7f41fc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221123769e1a89ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faace0b1640 1 --2- 172.21.15.50:0/3419431817 >> v2:172.21.15.50:3300/0 conn(0x7faac80afd80 0x7faac815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7f3681ffb640 1 -- 172.21.15.50:0/4173408412 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674037070 con 0x7f36840b7df0 2024-04-02T16:22:26.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faace0b1640 1 -- 172.21.15.50:0/3419431817 >> v2:172.21.15.188:3300/0 conn(0x7faac806e820 msgr2=0x7faac815ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7f3681ffb640 1 -- 172.21.15.50:0/4173408412 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674030710 con 0x7f36840b7df0 2024-04-02T16:22:26.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7f79a37fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f79a4145b50 0x7f79a40d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55056/0 (socket says 172.21.15.50:55056) 2024-04-02T16:22:26.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7f3681ffb640 1 -- 172.21.15.50:0/4173408412 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674030ae0 con 0x7f36840b7df0 2024-04-02T16:22:26.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faace0b1640 1 --2- 172.21.15.50:0/3419431817 >> v2:172.21.15.188:3300/0 conn(0x7faac806e820 0x7faac815ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a9d40640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f79a41604e0 0x7f79a40cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f420b7fe640 1 -- 172.21.15.50:0/3966579200 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc034070 con 0x7f420c06edb0 2024-04-02T16:22:26.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faace0b1640 1 -- 172.21.15.50:0/3419431817 >> v1:172.21.15.50:6789/0 conn(0x7faac80748d0 legacy=0x7faac80b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.402+0000 7faace0b1640 1 -- 172.21.15.50:0/3419431817 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac815e530 con 0x7faac80afd80 2024-04-02T16:22:26.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a9d40640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a4072f30 con 0x7f79a4145b50 2024-04-02T16:22:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7fb697fff640 1 --2- 172.21.15.50:0/3530121379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698162660 0x7fb698072f90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb690009d90 tx=0x7fb69002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f82f38efeb2c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a9d40640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a4160c90 con 0x7f79a40cda00 2024-04-02T16:22:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a9d40640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f79a40d0d40 con 0x7f79a41604e0 2024-04-02T16:22:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a3fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f79a41604e0 0x7f79a40cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a2ffd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f79a40cda00 0x7f79a415fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:44052/0 (socket says 172.21.15.50:44052) 2024-04-02T16:22:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a2ffd640 1 -- 172.21.15.50:0/3578668520 learned_addr learned my addr 172.21.15.50:0/3578668520 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:26.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f420b7fe640 1 -- 172.21.15.50:0/3966579200 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc030860 con 0x7f420c06edb0 2024-04-02T16:22:26.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f420b7fe640 1 -- 172.21.15.50:0/3966579200 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc030c30 con 0x7f420c06edb0 2024-04-02T16:22:26.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7fb696ffd640 1 -- 172.21.15.50:0/3530121379 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690039070 con 0x7fb698162660 2024-04-02T16:22:26.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7fb696ffd640 1 -- 172.21.15.50:0/3530121379 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69002cb40 con 0x7fb698162660 2024-04-02T16:22:26.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7fb696ffd640 1 -- 172.21.15.50:0/3530121379 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690034440 con 0x7fb698162660 2024-04-02T16:22:26.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f60ceffd640 1 --2- 172.21.15.50:0/606234982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c8049b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60c40097a0 tx=0x7f60c402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba61a5a65800e200 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a3fff640 1 -- 172.21.15.50:0/3578668520 >> v1:172.21.15.188:6789/0 conn(0x7f79a40cda00 legacy=0x7f79a415fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a3fff640 1 -- 172.21.15.50:0/3578668520 >> v1:172.21.15.50:6789/0 conn(0x7f79a4145b50 legacy=0x7f79a40d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.403+0000 7f79a3fff640 1 -- 172.21.15.50:0/3578668520 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a4145f30 con 0x7f79a41604e0 2024-04-02T16:22:26.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f60cdffb640 1 -- 172.21.15.50:0/606234982 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4034070 con 0x7f60c80e7a40 2024-04-02T16:22:26.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f60cdffb640 1 -- 172.21.15.50:0/606234982 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c4030610 con 0x7f60c80e7a40 2024-04-02T16:22:26.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7faace0b1640 1 --2- 172.21.15.50:0/3419431817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac815ed30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faac806c2f0 tx=0x7faac402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12efc4f62b4b27b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 -- 172.21.15.50:0/3966579200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06edb0 msgr2=0x7f420c0acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 --2- 172.21.15.50:0/3966579200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06edb0 0x7f420c0acf20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009870 tx=0x7f41fc02b9a0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f60cdffb640 1 -- 172.21.15.50:0/606234982 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40309e0 con 0x7f60c80e7a40 2024-04-02T16:22:26.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 -- 172.21.15.50:0/3966579200 shutdown_connections 2024-04-02T16:22:26.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 --2- 172.21.15.50:0/3966579200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06edb0 0x7f420c0acf20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 --2- 172.21.15.50:0/3966579200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c0d0790 0x7f420c15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 -- 172.21.15.50:0/3966579200 >> 172.21.15.50:0/3966579200 conn(0x7f420c06b520 msgr2=0x7f420c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 -- 172.21.15.50:0/3966579200 shutdown_connections 2024-04-02T16:22:26.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7fb69e338640 1 -- 172.21.15.50:0/3530121379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698162660 msgr2=0x7fb698072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7fb69e338640 1 --2- 172.21.15.50:0/3530121379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698162660 0x7fb698072f90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb690009d90 tx=0x7fb69002bd90 comp rx=0 tx=0).stop 2024-04-02T16:22:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7faacd0af640 1 -- 172.21.15.50:0/3419431817 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40317c0 con 0x7faac80afd80 2024-04-02T16:22:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f36893af640 1 -- 172.21.15.50:0/4173408412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 msgr2=0x7f3684166db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f36893af640 1 --2- 172.21.15.50:0/4173408412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f3684166db0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f36740097d0 tx=0x7f367402b6e0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7faacd0af640 1 -- 172.21.15.50:0/3419431817 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac4037070 con 0x7faac80afd80 2024-04-02T16:22:26.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7faacd0af640 1 -- 172.21.15.50:0/3419431817 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4031bc0 con 0x7faac80afd80 2024-04-02T16:22:26.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f36893af640 1 -- 172.21.15.50:0/4173408412 shutdown_connections 2024-04-02T16:22:26.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f60cffff640 1 -- 172.21.15.50:0/606234982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 msgr2=0x7f60c8049b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f60cffff640 1 --2- 172.21.15.50:0/606234982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c8049b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60c40097a0 tx=0x7f60c402b960 comp rx=0 tx=0).stop 2024-04-02T16:22:26.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7fb69e338640 1 -- 172.21.15.50:0/3530121379 shutdown_connections 2024-04-02T16:22:26.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7fb69e338640 1 --2- 172.21.15.50:0/3530121379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb698162660 0x7fb698072f90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7fb69e338640 1 -- 172.21.15.50:0/3530121379 >> 172.21.15.50:0/3530121379 conn(0x7fb69806b700 msgr2=0x7fb69806bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f96af426640 1 Processor -- start 2024-04-02T16:22:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7fb69e338640 1 -- 172.21.15.50:0/3530121379 shutdown_connections 2024-04-02T16:22:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f60cffff640 1 -- 172.21.15.50:0/606234982 shutdown_connections 2024-04-02T16:22:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f60cffff640 1 --2- 172.21.15.50:0/606234982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c8049b50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f36893af640 1 --2- 172.21.15.50:0/4173408412 >> v2:172.21.15.50:3300/0 conn(0x7f36840b32b0 0x7f36840b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f36893af640 1 --2- 172.21.15.50:0/4173408412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f3684166db0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f36893af640 1 -- 172.21.15.50:0/4173408412 >> 172.21.15.50:0/4173408412 conn(0x7f36840d0dc0 msgr2=0x7f36840d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f60cffff640 1 -- 172.21.15.50:0/606234982 >> 172.21.15.50:0/606234982 conn(0x7f60c8056cc0 msgr2=0x7f60c80570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f79a3fff640 1 --2- 172.21.15.50:0/3578668520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a41604e0 0x7f79a40cdeb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f799c009d90 tx=0x7f799c02bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b70649d92225f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.404+0000 7f4212d60640 1 -- 172.21.15.50:0/3966579200 wait complete. 2024-04-02T16:22:26.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f60cffff640 1 -- 172.21.15.50:0/606234982 shutdown_connections 2024-04-02T16:22:26.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f36893af640 1 -- 172.21.15.50:0/4173408412 shutdown_connections 2024-04-02T16:22:26.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f79a27fc640 1 -- 172.21.15.50:0/3578668520 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c02d920 con 0x7f79a41604e0 2024-04-02T16:22:26.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f79a27fc640 1 -- 172.21.15.50:0/3578668520 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c02dac0 con 0x7f79a41604e0 2024-04-02T16:22:26.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f79a27fc640 1 -- 172.21.15.50:0/3578668520 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c02c440 con 0x7f79a41604e0 2024-04-02T16:22:26.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7faad0315640 1 -- 172.21.15.50:0/3419431817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 msgr2=0x7faac815ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7faad0315640 1 --2- 172.21.15.50:0/3419431817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac815ed30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faac806c2f0 tx=0x7faac402b860 comp rx=0 tx=0).stop 2024-04-02T16:22:26.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f36893af640 1 -- 172.21.15.50:0/4173408412 wait complete. 2024-04-02T16:22:26.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f60cffff640 1 -- 172.21.15.50:0/606234982 wait complete. 2024-04-02T16:22:26.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7faad0315640 1 -- 172.21.15.50:0/3419431817 shutdown_connections 2024-04-02T16:22:26.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7f4212d60640 1 Processor -- start 2024-04-02T16:22:26.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7faad0315640 1 --2- 172.21.15.50:0/3419431817 >> v2:172.21.15.188:3300/0 conn(0x7faac806e820 0x7faac815ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7faad0315640 1 --2- 172.21.15.50:0/3419431817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac815ed30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7faad0315640 1 -- 172.21.15.50:0/3419431817 >> 172.21.15.50:0/3419431817 conn(0x7faac806b520 msgr2=0x7faac806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7faad0315640 1 -- 172.21.15.50:0/3419431817 shutdown_connections 2024-04-02T16:22:26.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7f96af426640 1 -- start start 2024-04-02T16:22:26.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7f96af426640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f96a00eacf0 0x7f96a004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7faad0315640 1 -- 172.21.15.50:0/3419431817 wait complete. 2024-04-02T16:22:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7f79a9d40640 1 -- 172.21.15.50:0/3578668520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a41604e0 msgr2=0x7f79a40cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7f79a9d40640 1 --2- 172.21.15.50:0/3578668520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a41604e0 0x7f79a40cdeb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f799c009d90 tx=0x7f799c02bbe0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7f96af426640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f96a0006e90 0x7f96a0007300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f944f59e640 1 Processor -- start 2024-04-02T16:22:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.405+0000 7fb69e338640 1 -- 172.21.15.50:0/3530121379 wait complete. 2024-04-02T16:22:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7fb69e338640 1 Processor -- start 2024-04-02T16:22:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f79a9d40640 1 -- 172.21.15.50:0/3578668520 shutdown_connections 2024-04-02T16:22:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f79a9d40640 1 --2- 172.21.15.50:0/3578668520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a41604e0 0x7f79a40cdeb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f79a9d40640 1 -- 172.21.15.50:0/3578668520 >> 172.21.15.50:0/3578668520 conn(0x7f79a406e6c0 msgr2=0x7f79a406eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96ae424640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f96a00eacf0 0x7f96a004fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96ae424640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f96a00eacf0 0x7f96a004fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42554/0 (socket says 172.21.15.50:42554) 2024-04-02T16:22:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96ae424640 1 -- 172.21.15.50:0/3550657172 learned_addr learned my addr 172.21.15.50:0/3550657172 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f944f59e640 1 -- start start 2024-04-02T16:22:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f60cffff640 1 Processor -- start 2024-04-02T16:22:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96aec25640 1 --2- 172.21.15.50:0/3550657172 >> v2:172.21.15.50:3300/0 conn(0x7f96a0006e90 0x7f96a0007300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7fb69e338640 1 -- start start 2024-04-02T16:22:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a0053710 con 0x7f96a004ffc0 2024-04-02T16:22:26.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.406+0000 7f4212d60640 1 -- start start 2024-04-02T16:22:26.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f96a0007840 con 0x7f96a0006e90 2024-04-02T16:22:26.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f96a0054320 con 0x7f96a00eacf0 2024-04-02T16:22:26.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f60cffff640 1 -- start start 2024-04-02T16:22:26.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f79a9d40640 1 -- 172.21.15.50:0/3578668520 shutdown_connections 2024-04-02T16:22:26.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f96ae424640 1 -- 172.21.15.50:0/3550657172 >> v2:172.21.15.50:3300/0 conn(0x7f96a0006e90 msgr2=0x7f96a0007300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4212d60640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06c010 0x7f420c0ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f96ae424640 1 --2- 172.21.15.50:0/3550657172 >> v2:172.21.15.50:3300/0 conn(0x7f96a0006e90 0x7f96a0007300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.407+0000 7f79a9d40640 1 -- 172.21.15.50:0/3578668520 wait complete. 2024-04-02T16:22:26.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4212d60640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c06edb0 0x7f420c0cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f944f59e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f94480501a0 0x7f9448054270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4212d60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f420c0d0790 0x7f420c144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f944f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4210afc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06c010 0x7f420c0ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4210afc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06c010 0x7f420c0ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45134/0 (socket says 172.21.15.50:45134) 2024-04-02T16:22:26.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f96ae424640 1 -- 172.21.15.50:0/3550657172 >> v1:172.21.15.188:6789/0 conn(0x7f96a004ffc0 legacy=0x7f96a0037bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:22:26.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f944f59e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9448006af0 con 0x7f94480df420 2024-04-02T16:22:26.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f944f59e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f94480068a0 con 0x7f94480501a0 2024-04-02T16:22:26.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f944f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9448006dd0 con 0x7f944800aaa0 2024-04-02T16:22:26.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7fb69e338640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69806d1f0 0x7fb698200d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7fb69e338640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6980b4760 0x7fb698204d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7fb69e338640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb698162660 0x7fb6981f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7fb69e338640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb69806dfb0 con 0x7fb6980b4760 2024-04-02T16:22:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7fb69e338640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6980b36d0 con 0x7fb69806d1f0 2024-04-02T16:22:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7fb69e338640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb6980ad480 con 0x7fb698162660 2024-04-02T16:22:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480382c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57202/0 (socket says 172.21.15.50:57202) 2024-04-02T16:22:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60c803c350 0x7f60c80e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60c804a090 0x7f60c804b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f944ed9d640 1 -- 172.21.15.50:0/497258263 learned_addr learned my addr 172.21.15.50:0/497258263 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7faad0315640 1 Processor -- start 2024-04-02T16:22:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4210afc640 1 -- 172.21.15.50:0/250092945 learned_addr learned my addr 172.21.15.50:0/250092945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f420c0b47c0 con 0x7f420c06c010 2024-04-02T16:22:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f420c071ea0 con 0x7f420c0d0790 2024-04-02T16:22:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0b4940 con 0x7f420c06edb0 2024-04-02T16:22:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f420bfff640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c06edb0 0x7f420c0cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f42112fd640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f420c0d0790 0x7f420c144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f944dd9b640 1 --2- 172.21.15.50:0/497258263 >> v2:172.21.15.50:3300/0 conn(0x7f94480501a0 0x7f9448054270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c804b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7faad0315640 1 -- start start 2024-04-02T16:22:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7fb6977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6980b4760 0x7fb698204d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cffff640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8054fa0 con 0x7f60c80e7a40 2024-04-02T16:22:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.408+0000 7f96ae424640 1 -- 172.21.15.50:0/3550657172 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a00539f0 con 0x7f96a00eacf0 2024-04-02T16:22:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cffff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60c804a5a0 con 0x7f60c803c350 2024-04-02T16:22:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60c80de160 con 0x7f60c804a090 2024-04-02T16:22:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f96aec25640 1 --2- 172.21.15.50:0/3550657172 >> v2:172.21.15.50:3300/0 conn(0x7f96a0006e90 0x7f96a0007300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7fb6977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6980b4760 0x7fb698204d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45148/0 (socket says 172.21.15.50:45148) 2024-04-02T16:22:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7fb6977fe640 1 -- 172.21.15.50:0/2028188180 learned_addr learned my addr 172.21.15.50:0/2028188180 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f79a9d40640 1 Processor -- start 2024-04-02T16:22:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f944dd9b640 1 -- 172.21.15.50:0/497258263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 msgr2=0x7f94480382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f944dd9b640 1 --2- 172.21.15.50:0/497258263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f96ae424640 1 --2- 172.21.15.50:0/3550657172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a00eacf0 0x7f96a004fa80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96a806b390 tx=0x7f96a80a6e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf4133fe4690881 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f944dd9b640 1 -- 172.21.15.50:0/497258263 >> v1:172.21.15.50:6789/0 conn(0x7f94480df420 legacy=0x7f944804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7fb697fff640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69806d1f0 0x7fb698200d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f944dd9b640 1 -- 172.21.15.50:0/497258263 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94480070b0 con 0x7f94480501a0 2024-04-02T16:22:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7fb69c8d5640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb698162660 0x7fb6981f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb6977fe640 1 -- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb698162660 msgr2=0x7fb6981f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb6977fe640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb698162660 0x7fb6981f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f96ad422640 1 -- 172.21.15.50:0/3550657172 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80b4070 con 0x7f96a00eacf0 2024-04-02T16:22:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f96ad422640 1 -- 172.21.15.50:0/3550657172 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80a7ce0 con 0x7f96a00eacf0 2024-04-02T16:22:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f96ad422640 1 -- 172.21.15.50:0/3550657172 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80af640 con 0x7f96a00eacf0 2024-04-02T16:22:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb6977fe640 1 -- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69806d1f0 msgr2=0x7fb698200d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb6977fe640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69806d1f0 0x7fb698200d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb6977fe640 1 -- 172.21.15.50:0/2028188180 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb698205290 con 0x7fb6980b4760 2024-04-02T16:22:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb6977fe640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6980b4760 0x7fb698204d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c460 tx=0x7fb68c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7faad0315640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac806e820 0x7faac81516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c804b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c804b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45162/0 (socket says 172.21.15.50:45162) 2024-04-02T16:22:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f60cf7fe640 1 -- 172.21.15.50:0/2982424279 learned_addr learned my addr 172.21.15.50:0/2982424279 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f60cf7fe640 1 -- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60c804a090 msgr2=0x7f60c804b190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T16:22:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f60cf7fe640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60c804a090 0x7f60c804b190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f60cf7fe640 1 -- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60c803c350 msgr2=0x7f60c80e01f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T16:22:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f60cf7fe640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60c803c350 0x7f60c80e01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f60cf7fe640 1 -- 172.21.15.50:0/2982424279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c804bc10 con 0x7f60c80e7a40 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f60cf7fe640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c804b6d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f60c000c400 tx=0x7f60c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f36893af640 1 Processor -- start 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f36893af640 1 -- start start 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f36893af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36840b32b0 0x7f3684160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f36893af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36840b5dc0 0x7f3684163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f36893af640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f36840a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f36893af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f36840b81d0 con 0x7f36840b5dc0 2024-04-02T16:22:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f36893af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3684166890 con 0x7f36840b32b0 2024-04-02T16:22:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f36893af640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36840b6250 con 0x7f36840b7df0 2024-04-02T16:22:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7faad0315640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac80748d0 0x7faac8140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.409+0000 7f79a9d40640 1 -- start start 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a00eacf0 msgr2=0x7f96a004fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faace0b1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac806e820 0x7faac81516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faad0315640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac81410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faad0315640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faac80b0160 con 0x7faac80afd80 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faad0315640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faac815f270 con 0x7faac80748d0 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faace0b1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac806e820 0x7faac81516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57226/0 (socket says 172.21.15.50:57226) 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faad0315640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faac80b02e0 con 0x7faac806e820 2024-04-02T16:22:26.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faace0b1640 1 -- 172.21.15.50:0/3757353566 learned_addr learned my addr 172.21.15.50:0/3757353566 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f96af426640 1 --2- 172.21.15.50:0/3550657172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a00eacf0 0x7f96a004fa80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96a806b390 tx=0x7f96a80a6e30 comp rx=0 tx=0).stop 2024-04-02T16:22:26.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f60b7fff640 1 -- 172.21.15.50:0/2982424279 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0017070 con 0x7f60c80e7a40 2024-04-02T16:22:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a9d40640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a40cda00 0x7f79a42009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb694ff9640 1 -- 172.21.15.50:0/2028188180 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c017070 con 0x7fb6980b4760 2024-04-02T16:22:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7fb694ff9640 1 -- 172.21.15.50:0/2028188180 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c005d00 con 0x7fb6980b4760 2024-04-02T16:22:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faacd8b0640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac80748d0 0x7faac8140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 shutdown_connections 2024-04-02T16:22:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f96af426640 1 --2- 172.21.15.50:0/3550657172 >> v2:172.21.15.50:3300/0 conn(0x7f96a0006e90 0x7f96a0007300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f96af426640 1 --2- 172.21.15.50:0/3550657172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a00eacf0 0x7f96a004fa80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 >> 172.21.15.50:0/3550657172 conn(0x7f96a0009fa0 msgr2=0x7f96a000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7faace8b2640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac81410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f36840a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f36840a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57224/0 (socket says 172.21.15.50:57224) 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 -- 172.21.15.50:0/3722572953 learned_addr learned my addr 172.21.15.50:0/3722572953 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3682ffd640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36840b32b0 0x7f3684160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f36827fc640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36840b5dc0 0x7f3684163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.410+0000 7f42112fd640 1 -- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c06edb0 msgr2=0x7f420c0cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f42112fd640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c06edb0 0x7f420c0cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f42112fd640 1 -- 172.21.15.50:0/250092945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06c010 msgr2=0x7f420c0ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f42112fd640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06c010 0x7f420c0ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f42112fd640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c0cca10 con 0x7f420c0d0790 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a9d40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a4145b50 0x7f79a415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a9d40640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79a41604e0 0x7f79a415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a9d40640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40cfe30 con 0x7f79a4145b50 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a9d40640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40cfb50 con 0x7f79a40cda00 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a9d40640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f79a40d0110 con 0x7f79a41604e0 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a40cda00 0x7f79a42009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a40cda00 0x7f79a42009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42598/0 (socket says 172.21.15.50:42598) 2024-04-02T16:22:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a37fe640 1 -- 172.21.15.50:0/1515921899 learned_addr learned my addr 172.21.15.50:0/1515921899 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f944dd9b640 1 --2- 172.21.15.50:0/497258263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480501a0 0x7f9448054270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94400097a0 tx=0x7f944002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ba3d52c13aff22 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f42112fd640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f420c0d0790 0x7f420c144df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41fc033000 tx=0x7f41fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a2ffd640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a4145b50 0x7f79a415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f79a3fff640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79a41604e0 0x7f79a415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 -- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36840b32b0 msgr2=0x7f3684160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36840b32b0 0x7f3684160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 -- 172.21.15.50:0/3722572953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36840b5dc0 msgr2=0x7f3684163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36840b5dc0 0x7f3684163f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f3683fff640 1 -- 172.21.15.50:0/3722572953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36840a1980 con 0x7f36840b7df0 2024-04-02T16:22:26.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f42097fa640 1 -- 172.21.15.50:0/250092945 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc034040 con 0x7f420c0d0790 2024-04-02T16:22:26.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7fb694ff9640 1 -- 172.21.15.50:0/2028188180 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c007310 con 0x7fb6980b4760 2024-04-02T16:22:26.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c144310 con 0x7f420c0d0790 2024-04-02T16:22:26.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f944d59a640 1 -- 172.21.15.50:0/497258263 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440035020 con 0x7f94480501a0 2024-04-02T16:22:26.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f944d59a640 1 -- 172.21.15.50:0/497258263 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9440031830 con 0x7f94480501a0 2024-04-02T16:22:26.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f944d59a640 1 -- 172.21.15.50:0/497258263 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440035020 con 0x7f94480501a0 2024-04-02T16:22:26.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c0cc700 con 0x7f420c0d0790 2024-04-02T16:22:26.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb698201360 con 0x7fb6980b4760 2024-04-02T16:22:26.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb698201920 con 0x7fb6980b4760 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f42097fa640 1 -- 172.21.15.50:0/250092945 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc0433f0 con 0x7f420c0d0790 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f42097fa640 1 -- 172.21.15.50:0/250092945 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0437e0 con 0x7f420c0d0790 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f79a37fe640 1 -- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79a41604e0 msgr2=0x7f79a415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f79a37fe640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79a41604e0 0x7f79a415d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f79a37fe640 1 -- 172.21.15.50:0/1515921899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a4145b50 msgr2=0x7f79a415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f79a37fe640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a4145b50 0x7f79a415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f79a37fe640 1 -- 172.21.15.50:0/1515921899 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a41403e0 con 0x7f79a40cda00 2024-04-02T16:22:26.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f79a3fff640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79a41604e0 0x7f79a415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7faace8b2640 1 -- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac806e820 msgr2=0x7faac81516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7faace8b2640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac806e820 0x7faac81516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7faace8b2640 1 -- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac80748d0 msgr2=0x7faac8140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7faace8b2640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac80748d0 0x7faac8140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7faace8b2640 1 -- 172.21.15.50:0/3757353566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac8141b20 con 0x7faac80afd80 2024-04-02T16:22:26.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f79a37fe640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a40cda00 0x7f79a42009a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f799800c7c0 tx=0x7f799800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7faace8b2640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac81410e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faac000c730 tx=0x7faac000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f3683fff640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f36840a0f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f367c00caf0 tx=0x7f367c0020d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f366b7fe640 1 -- 172.21.15.50:0/3722572953 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c017070 con 0x7f36840b7df0 2024-04-02T16:22:26.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f36893af640 1 -- 172.21.15.50:0/3722572953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36840c8d30 con 0x7f36840b7df0 2024-04-02T16:22:26.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f36893af640 1 -- 172.21.15.50:0/3722572953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f36840c92f0 con 0x7f36840b7df0 2024-04-02T16:22:26.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 shutdown_connections 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c804beb0 con 0x7f60c80e7a40 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.411+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c804e490 con 0x7f60c80e7a40 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f60b7fff640 1 -- 172.21.15.50:0/2982424279 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c0005ce0 con 0x7f60c80e7a40 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.412+0000 7f60b7fff640 1 -- 172.21.15.50:0/2982424279 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00072d0 con 0x7f60c80e7a40 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f798bfff640 1 -- 172.21.15.50:0/1515921899 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998018070 con 0x7f79a40cda00 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f798bfff640 1 -- 172.21.15.50:0/1515921899 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79980074e0 con 0x7f79a40cda00 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f798bfff640 1 -- 172.21.15.50:0/1515921899 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998007690 con 0x7f79a40cda00 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7f79a9d40640 1 -- 172.21.15.50:0/1515921899 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a41406c0 con 0x7f79a40cda00 2024-04-02T16:22:26.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7f79a9d40640 1 -- 172.21.15.50:0/1515921899 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a4140f60 con 0x7f79a40cda00 2024-04-02T16:22:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7faabeffd640 1 -- 172.21.15.50:0/3757353566 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0016020 con 0x7faac80afd80 2024-04-02T16:22:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7faabeffd640 1 -- 172.21.15.50:0/3757353566 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac0006d50 con 0x7faac80afd80 2024-04-02T16:22:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7faabeffd640 1 -- 172.21.15.50:0/3757353566 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0003bb0 con 0x7faac80afd80 2024-04-02T16:22:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7faad0315640 1 -- 172.21.15.50:0/3757353566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac8141e00 con 0x7faac80afd80 2024-04-02T16:22:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7faad0315640 1 -- 172.21.15.50:0/3757353566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac8151c20 con 0x7faac80afd80 2024-04-02T16:22:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec005290 con 0x7f420c0d0790 2024-04-02T16:22:26.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7f42097fa640 1 -- 172.21.15.50:0/250092945 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41fc04a060 con 0x7f420c0d0790 2024-04-02T16:22:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7fb694ff9640 1 -- 172.21.15.50:0/2028188180 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb68c01a070 con 0x7fb6980b4760 2024-04-02T16:22:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7fb694ff9640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb68406c920 0x7fb68406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7fb697fff640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb68406c920 0x7fb68406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7fb694ff9640 1 -- 172.21.15.50:0/2028188180 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb68c08b270 con 0x7fb6980b4760 2024-04-02T16:22:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7f366b7fe640 1 -- 172.21.15.50:0/3722572953 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f367c003df0 con 0x7f36840b7df0 2024-04-02T16:22:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f366b7fe640 1 -- 172.21.15.50:0/3722572953 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c00a940 con 0x7f36840b7df0 2024-04-02T16:22:26.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b0005290 con 0x7f60c80e7a40 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7f944f59e640 1 -- 172.21.15.50:0/497258263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480501a0 msgr2=0x7f9448054270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7f944f59e640 1 --2- 172.21.15.50:0/497258263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480501a0 0x7f9448054270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94400097a0 tx=0x7f944002b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f944f59e640 1 -- 172.21.15.50:0/497258263 shutdown_connections 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f944f59e640 1 --2- 172.21.15.50:0/497258263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f944f59e640 1 --2- 172.21.15.50:0/497258263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480501a0 0x7f9448054270 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94400097a0 tx=0x7f944002b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f944f59e640 1 -- 172.21.15.50:0/497258263 >> 172.21.15.50:0/497258263 conn(0x7f9448009fb0 msgr2=0x7f944800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.413+0000 7f96af426640 1 -- 172.21.15.50:0/3550657172 wait complete. 2024-04-02T16:22:26.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7f96af426640 1 Processor -- start 2024-04-02T16:22:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7f96af426640 1 -- start start 2024-04-02T16:22:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.415+0000 7f944f59e640 1 -- 172.21.15.50:0/497258263 shutdown_connections 2024-04-02T16:22:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac815eca0 con 0x7faac80afd80 2024-04-02T16:22:26.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f42097fa640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f41f806c9f0 0x7f41f806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.414+0000 7f36893af640 1 -- 172.21.15.50:0/3722572953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3664005290 con 0x7f36840b7df0 2024-04-02T16:22:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f366b7fe640 1 -- 172.21.15.50:0/3722572953 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f367c01a070 con 0x7f36840b7df0 2024-04-02T16:22:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f4210afc640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f41f806c9f0 0x7f41f806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f96af426640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a0006e90 0x7f96a002b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f96af426640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96a004ffc0 0x7f96a00e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f366b7fe640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f367006c9f0 0x7f367006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f96ae424640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a0006e90 0x7f96a002b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96ae424640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a0006e90 0x7f96a002b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42604/0 (socket says 172.21.15.50:42604) 2024-04-02T16:22:26.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f60b7fff640 1 -- 172.21.15.50:0/2982424279 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60c001a070 con 0x7f60c80e7a40 2024-04-02T16:22:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96ae424640 1 -- 172.21.15.50:0/1143828829 learned_addr learned my addr 172.21.15.50:0/1143828829 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f366b7fe640 1 -- 172.21.15.50:0/3722572953 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f367c08b4d0 con 0x7f36840b7df0 2024-04-02T16:22:26.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96af426640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96a00eacf0 0x7f96a00e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f42097fa640 1 -- 172.21.15.50:0/250092945 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f41fc0adfa0 con 0x7f420c0d0790 2024-04-02T16:22:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96a00547b0 con 0x7f96a004ffc0 2024-04-02T16:22:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96adc23640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96a004ffc0 0x7f96a00e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0053fb0 con 0x7f96a0006e90 2024-04-02T16:22:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f96a0054930 con 0x7f96a00eacf0 2024-04-02T16:22:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96aec25640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96a00eacf0 0x7f96a00e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7f944f59e640 1 -- 172.21.15.50:0/497258263 wait complete. 2024-04-02T16:22:26.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a4140890 con 0x7f79a40cda00 2024-04-02T16:22:26.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.416+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6980756f0 con 0x7fb6980b4760 2024-04-02T16:22:26.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7faabeffd640 1 -- 172.21.15.50:0/3757353566 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faac00050d0 con 0x7faac80afd80 2024-04-02T16:22:26.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f60b7fff640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f60bc06c9c0 0x7f60bc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f4210afc640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f41f806c9f0 0x7f41f806eeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f420c1554d0 tx=0x7f4200009110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7fb697fff640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb68406c920 0x7fb68406ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb690036000 tx=0x7fb690030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f60b7fff640 1 -- 172.21.15.50:0/2982424279 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f60c008b370 con 0x7f60c80e7a40 2024-04-02T16:22:26.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f3682ffd640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f367006c9f0 0x7f367006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f60ceffd640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f60bc06c9c0 0x7f60bc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96ae424640 1 -- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96a00eacf0 msgr2=0x7f96a00e9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96ae424640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96a00eacf0 0x7f96a00e9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96ae424640 1 -- 172.21.15.50:0/1143828829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96a004ffc0 msgr2=0x7f96a00e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96ae424640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96a004ffc0 0x7f96a00e96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7faabeffd640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7faaac06c9c0 0x7faaac06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.417+0000 7f96ae424640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a002bc90 con 0x7f96a0006e90 2024-04-02T16:22:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7faabeffd640 1 -- 172.21.15.50:0/3757353566 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faac008bcb0 con 0x7faac80afd80 2024-04-02T16:22:26.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f96ae424640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a0006e90 0x7f96a002b460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f96a802d900 tx=0x7f96a806b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f944f59e640 1 Processor -- start 2024-04-02T16:22:26.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f944f59e640 1 -- start start 2024-04-02T16:22:26.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7faace0b1640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7faaac06c9c0 0x7faaac06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f3682ffd640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f367006c9f0 0x7f367006eeb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3674034000 tx=0x7f3674008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f60ceffd640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f60bc06c9c0 0x7f60bc06ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f60c4036000 tx=0x7f60c4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7faace0b1640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7faaac06c9c0 0x7faaac06ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faac40091d0 tx=0x7faac403d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f944f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f944f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94480df420 0x7f94480d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.418+0000 7f944f59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480d4c20 0x7f94480d6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944f59e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9448052b40 con 0x7f94480d4c20 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94480547b0 con 0x7f94480df420 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9448052cc0 con 0x7f944800aaa0 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480d4c20 0x7f94480d6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944e59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480d4c20 0x7f94480d6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45212/0 (socket says 172.21.15.50:45212) 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 -- 172.21.15.50:0/585516252 learned_addr learned my addr 172.21.15.50:0/585516252 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944dd9b640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94480df420 0x7f94480d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 -- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 msgr2=0x7f94480ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 -- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94480df420 msgr2=0x7f94480d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94480df420 0x7f94480d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 -- 172.21.15.50:0/585516252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94480d6160 con 0x7f94480d4c20 2024-04-02T16:22:26.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.419+0000 7f944ed9d640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480d4c20 0x7f94480d6890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9450050ad0 tx=0x7f94500759a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.420+0000 7f42097fa640 1 -- 172.21.15.50:0/250092945 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f41fc03e070 con 0x7f420c0d0790 2024-04-02T16:22:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.420+0000 7f969effd640 1 -- 172.21.15.50:0/1143828829 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80b4040 con 0x7f96a0006e90 2024-04-02T16:22:26.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.420+0000 7f969effd640 1 -- 172.21.15.50:0/1143828829 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80b9670 con 0x7f96a0006e90 2024-04-02T16:22:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.420+0000 7f969effd640 1 -- 172.21.15.50:0/1143828829 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80b9aa0 con 0x7f96a0006e90 2024-04-02T16:22:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.420+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a002bf70 con 0x7f96a0006e90 2024-04-02T16:22:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.420+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a002c530 con 0x7f96a0006e90 2024-04-02T16:22:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.421+0000 7f366b7fe640 1 -- 172.21.15.50:0/3722572953 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f367c058fe0 con 0x7f36840b7df0 2024-04-02T16:22:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.421+0000 7f798bfff640 1 -- 172.21.15.50:0/1515921899 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f799801b070 con 0x7f79a40cda00 2024-04-02T16:22:26.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.421+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a0005380 con 0x7f96a0006e90 2024-04-02T16:22:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.422+0000 7f798bfff640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f799006c970 0x7f799006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.422+0000 7f798bfff640 1 -- 172.21.15.50:0/1515921899 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f799808b7f0 con 0x7f79a40cda00 2024-04-02T16:22:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.422+0000 7f79a2ffd640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f799006c970 0x7f799006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.422+0000 7f79a2ffd640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f799006c970 0x7f799006ee30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79a413f3f0 tx=0x7f7994007e20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.423+0000 7fb694ff9640 1 -- 172.21.15.50:0/2028188180 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb698205290 con 0x7fb6980b4760 2024-04-02T16:22:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.425+0000 7f798bfff640 1 -- 172.21.15.50:0/1515921899 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7998022020 con 0x7f79a40cda00 2024-04-02T16:22:26.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.427+0000 7f969effd640 1 -- 172.21.15.50:0/1143828829 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96a80c3070 con 0x7f96a0006e90 2024-04-02T16:22:26.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.427+0000 7f969effd640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.427+0000 7f969effd640 1 -- 172.21.15.50:0/1143828829 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f96a81289d0 con 0x7f96a0006e90 2024-04-02T16:22:26.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.428+0000 7f96adc23640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.428+0000 7f60b7fff640 1 -- 172.21.15.50:0/2982424279 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f60c0059440 con 0x7f60c80e7a40 2024-04-02T16:22:26.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.430+0000 7f96adc23640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96a0047680 tx=0x7f96a40090e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.431+0000 7f969effd640 1 -- 172.21.15.50:0/1143828829 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f96a80f6510 con 0x7f96a0006e90 2024-04-02T16:22:26.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.434+0000 7f943effd640 1 -- 172.21.15.50:0/585516252 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945006c8a0 con 0x7f94480d4c20 2024-04-02T16:22:26.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.434+0000 7faabeffd640 1 -- 172.21.15.50:0/3757353566 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faac815eca0 con 0x7faac80afd80 2024-04-02T16:22:26.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.434+0000 7f943effd640 1 -- 172.21.15.50:0/585516252 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945006ca40 con 0x7f94480d4c20 2024-04-02T16:22:26.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.435+0000 7f943effd640 1 -- 172.21.15.50:0/585516252 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945006ce50 con 0x7f94480d4c20 2024-04-02T16:22:26.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.435+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94480cd060 con 0x7f94480d4c20 2024-04-02T16:22:26.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.435+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94480d53a0 con 0x7f94480d4c20 2024-04-02T16:22:26.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.437+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94480089d0 con 0x7f94480d4c20 2024-04-02T16:22:26.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.439+0000 7f943effd640 1 -- 172.21.15.50:0/585516252 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94500748d0 con 0x7f94480d4c20 2024-04-02T16:22:26.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.440+0000 7f943effd640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f943006cae0 0x7f943006efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:26.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.442+0000 7f943effd640 1 -- 172.21.15.50:0/585516252 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f94500d2e00 con 0x7f94480d4c20 2024-04-02T16:22:26.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.444+0000 7f944e59c640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f943006cae0 0x7f943006efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:26.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.448+0000 7f943effd640 1 -- 172.21.15.50:0/585516252 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f94500d4d60 con 0x7f94480d4c20 2024-04-02T16:22:26.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.483+0000 7f944e59c640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f943006cae0 0x7f943006efa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9444000f60 tx=0x7f9444009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:26.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.848+0000 7f36893af640 1 -- 172.21.15.50:0/3722572953 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3664005580 con 0x7f36840b7df0 2024-04-02T16:22:26.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.848+0000 7f366b7fe640 1 -- 172.21.15.50:0/3722572953 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f367c05d2e0 con 0x7f36840b7df0 2024-04-02T16:22:26.850 INFO:teuthology.orchestra.run.smithi050.stdout:55834574856 2024-04-02T16:22:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 -- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f367006c9f0 msgr2=0x7f367006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f41ec005850 con 0x7f420c0d0790 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f367006c9f0 0x7f367006eeb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3674034000 tx=0x7f3674008030 comp rx=0 tx=0).stop 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 -- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 msgr2=0x7f36840a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f36840a0f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f367c00caf0 tx=0x7f367c0020d0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 -- 172.21.15.50:0/3722572953 shutdown_connections 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36840b7df0 0x7f36840a0f00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f367006c9f0 0x7f367006eeb0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3674034000 tx=0x7f3674008030 comp rx=0 tx=0).stop 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36840b5dc0 0x7f3684163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 --2- 172.21.15.50:0/3722572953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36840b32b0 0x7f3684160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.852+0000 7f36697fa640 1 -- 172.21.15.50:0/3722572953 >> 172.21.15.50:0/3722572953 conn(0x7f36840d0dc0 msgr2=0x7f36841621f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.853+0000 7f36697fa640 1 -- 172.21.15.50:0/3722572953 shutdown_connections 2024-04-02T16:22:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.853+0000 7f36697fa640 1 -- 172.21.15.50:0/3722572953 wait complete. 2024-04-02T16:22:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.853+0000 7f42097fa640 1 -- 172.21.15.50:0/250092945 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41fc07bab0 con 0x7f420c0d0790 2024-04-02T16:22:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.853+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f79a4075760 con 0x7f79a40cda00 2024-04-02T16:22:26.856 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.854+0000 7f798bfff640 1 -- 172.21.15.50:0/1515921899 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f79a4072f30 con 0x7f79a40cda00 2024-04-02T16:22:26.857 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.856+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f41f806c9f0 msgr2=0x7f41f806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.856+0000 7f4212d60640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f41f806c9f0 0x7f41f806eeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f420c1554d0 tx=0x7f4200009110 comp rx=0 tx=0).stop 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.856+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f420c0d0790 msgr2=0x7f420c144df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.856+0000 7f4212d60640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f420c0d0790 0x7f420c144df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41fc033000 tx=0x7f41fc0046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 shutdown_connections 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f420c0d0790 0x7f420c144df0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f420c06edb0 0x7f420c0cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f41f806c9f0 0x7f41f806eeb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 --2- 172.21.15.50:0/250092945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f420c06c010 0x7f420c0ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 >> 172.21.15.50:0/250092945 conn(0x7f420c06b520 msgr2=0x7f420c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 shutdown_connections 2024-04-02T16:22:26.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f799006c970 msgr2=0x7f799006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f7989ffb640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f799006c970 0x7f799006ee30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f79a413f3f0 tx=0x7f7994007e20 comp rx=0 tx=0).stop 2024-04-02T16:22:26.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a40cda00 msgr2=0x7f79a42009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f7989ffb640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a40cda00 0x7f79a42009a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f799800c7c0 tx=0x7f799800cc80 comp rx=0 tx=0).stop 2024-04-02T16:22:26.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f4212d60640 1 -- 172.21.15.50:0/250092945 wait complete. 2024-04-02T16:22:26.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.857+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 shutdown_connections 2024-04-02T16:22:26.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.858+0000 7f7989ffb640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79a41604e0 0x7f79a415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.858+0000 7f7989ffb640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f799006c970 0x7f799006ee30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.858+0000 7f7989ffb640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79a4145b50 0x7f79a415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.858+0000 7f7989ffb640 1 --2- 172.21.15.50:0/1515921899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a40cda00 0x7f79a42009a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.858+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 >> 172.21.15.50:0/1515921899 conn(0x7f79a406e6c0 msgr2=0x7f79a40d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.858+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 shutdown_connections 2024-04-02T16:22:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.858+0000 7f7989ffb640 1 -- 172.21.15.50:0/1515921899 wait complete. 2024-04-02T16:22:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.861+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb6980ae650 con 0x7fb6980b4760 2024-04-02T16:22:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.862+0000 7fb694ff9640 1 -- 172.21.15.50:0/2028188180 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb68c059350 con 0x7fb6980b4760 2024-04-02T16:22:26.863 INFO:teuthology.orchestra.run.smithi050.stdout:55834574856 2024-04-02T16:22:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.864+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f96a0034a70 con 0x7f96a0006e90 2024-04-02T16:22:26.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.865+0000 7f969effd640 1 -- 172.21.15.50:0/1143828829 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96a80fa810 con 0x7f96a0006e90 2024-04-02T16:22:26.866 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.866+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb68406c920 msgr2=0x7fb68406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.866+0000 7fb69e338640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb68406c920 0x7fb68406ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb690036000 tx=0x7fb690030000 comp rx=0 tx=0).stop 2024-04-02T16:22:26.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.866+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6980b4760 msgr2=0x7fb698204d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.866+0000 7fb69e338640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6980b4760 0x7fb698204d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c460 tx=0x7fb68c00c920 comp rx=0 tx=0).stop 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 shutdown_connections 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb698162660 0x7fb6981f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb68406c920 0x7fb68406ede0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6980b4760 0x7fb698204d50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 --2- 172.21.15.50:0/2028188180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69806d1f0 0x7fb698200d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 >> 172.21.15.50:0/2028188180 conn(0x7fb69806b700 msgr2=0x7fb6980b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 shutdown_connections 2024-04-02T16:22:26.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.867+0000 7fb69e338640 1 -- 172.21.15.50:0/2028188180 wait complete. 2024-04-02T16:22:26.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.868+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f968c06c8f0 msgr2=0x7f968c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.868+0000 7f96af426640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96a0047680 tx=0x7f96a40090e0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.868+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a0006e90 msgr2=0x7f96a002b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.868+0000 7f96af426640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a0006e90 0x7f96a002b460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f96a802d900 tx=0x7f96a806b3e0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 shutdown_connections 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96a00eacf0 0x7f96a00e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96a004ffc0 0x7f96a00e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 --2- 172.21.15.50:0/1143828829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96a0006e90 0x7f96a002b460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 >> 172.21.15.50:0/1143828829 conn(0x7f96a0009fa0 msgr2=0x7f96a004a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 shutdown_connections 2024-04-02T16:22:26.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.869+0000 7f96af426640 1 -- 172.21.15.50:0/1143828829 wait complete. 2024-04-02T16:22:26.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.871+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7faac8075760 con 0x7faac80afd80 2024-04-02T16:22:26.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.872+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f60b0005580 con 0x7f60c80e7a40 2024-04-02T16:22:26.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.872+0000 7faabeffd640 1 -- 172.21.15.50:0/3757353566 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faac0020020 con 0x7faac80afd80 2024-04-02T16:22:26.874 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-04-02T16:22:26.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.873+0000 7f60b7fff640 1 -- 172.21.15.50:0/2982424279 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f60c0058d40 con 0x7f60c80e7a40 2024-04-02T16:22:26.876 INFO:teuthology.orchestra.run.smithi050.stdout:55834574856 2024-04-02T16:22:26.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-04-02T16:22:26.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7faaac06c9c0 msgr2=0x7faaac06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7faaac06c9c0 0x7faaac06ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faac40091d0 tx=0x7faac403d040 comp rx=0 tx=0).stop 2024-04-02T16:22:26.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 msgr2=0x7faac81410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac81410e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faac000c730 tx=0x7faac000cbf0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 shutdown_connections 2024-04-02T16:22:26.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7faaac06c9c0 0x7faaac06ee80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faac80afd80 0x7faac81410e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac80748d0 0x7faac8140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.876+0000 7faabcff9640 1 --2- 172.21.15.50:0/3757353566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac806e820 0x7faac81516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 >> 172.21.15.50:0/3757353566 conn(0x7faac806b520 msgr2=0x7faac815bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f60bc06c9c0 msgr2=0x7f60bc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f60bc06c9c0 0x7f60bc06ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f60c4036000 tx=0x7f60c4008030 comp rx=0 tx=0).stop 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 msgr2=0x7f60c804b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c804b6d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f60c000c400 tx=0x7f60c000c8c0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 shutdown_connections 2024-04-02T16:22:26.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7faabcff9640 1 -- 172.21.15.50:0/3757353566 wait complete. 2024-04-02T16:22:26.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 shutdown_connections 2024-04-02T16:22:26.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f60bc06c9c0 0x7f60bc06ee80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60c80e7a40 0x7f60c804b6d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60c804a090 0x7f60c804b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 --2- 172.21.15.50:0/2982424279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60c803c350 0x7f60c80e01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 >> 172.21.15.50:0/2982424279 conn(0x7f60c8056cc0 msgr2=0x7f60c8007b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 shutdown_connections 2024-04-02T16:22:26.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f94480dc5d0 con 0x7f94480d4c20 2024-04-02T16:22:26.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.877+0000 7f60cffff640 1 -- 172.21.15.50:0/2982424279 wait complete. 2024-04-02T16:22:26.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.878+0000 7f943effd640 1 -- 172.21.15.50:0/585516252 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f94500d9060 con 0x7f94480d4c20 2024-04-02T16:22:26.882 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:26.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f943006cae0 msgr2=0x7f943006efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f943006cae0 0x7f943006efa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9444000f60 tx=0x7f9444009040 comp rx=0 tx=0).stop 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480d4c20 msgr2=0x7f94480d6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480d4c20 0x7f94480d6890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9450050ad0 tx=0x7f94500759a0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 shutdown_connections 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f943006cae0 0x7f943006efa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94480d4c20 0x7f94480d6890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94480df420 0x7f94480d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 --2- 172.21.15.50:0/585516252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f944800aaa0 0x7f94480ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:26.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 >> 172.21.15.50:0/585516252 conn(0x7f9448009fb0 msgr2=0x7f9448034520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:26.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 shutdown_connections 2024-04-02T16:22:26.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:26.881+0000 7f944f59e640 1 -- 172.21.15.50:0/585516252 wait complete. 2024-04-02T16:22:26.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-04-02T16:22:26.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-04-02T16:22:26.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-04-02T16:22:26.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-04-02T16:22:26.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-04-02T16:22:27.876 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-02T16:22:27.878 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-02T16:22:27.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-02T16:22:27.891 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-02T16:22:27.892 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-02T16:22:27.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-02T16:22:27.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-02T16:22:27.901 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-02T16:22:28.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.039+0000 7fbfc359e640 1 Processor -- start 2024-04-02T16:22:28.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.039+0000 7fbfc359e640 1 -- start start 2024-04-02T16:22:28.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbfc40d0770 0x7fbfc40cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc359e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbfc415a220 0x7fbfc4071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc359e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc406fad0 con 0x7fbfc40ce230 2024-04-02T16:22:28.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc359e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc40cf850 con 0x7fbfc415a220 2024-04-02T16:22:28.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc359e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc40c4c70 con 0x7fbfc40d0770 2024-04-02T16:22:28.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc2d9d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fbfc40ce230 0x7fbfc40cac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:44070/0 (socket says 172.21.15.50:44070) 2024-04-02T16:22:28.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc2d9d640 1 -- 172.21.15.50:0/3758022978 learned_addr learned my addr 172.21.15.50:0/3758022978 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:28.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.041+0000 7fbfc1d9b640 1 --2- 172.21.15.50:0/3758022978 >> v2:172.21.15.50:3300/0 conn(0x7fbfc415a220 0x7fbfc4071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.042+0000 7fbfc1d9b640 1 -- 172.21.15.50:0/3758022978 >> v2:172.21.15.188:3300/0 conn(0x7fbfc40d0770 msgr2=0x7fbfc40cf150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T16:22:28.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.042+0000 7fbfc1d9b640 1 --2- 172.21.15.50:0/3758022978 >> v2:172.21.15.188:3300/0 conn(0x7fbfc40d0770 0x7fbfc40cf150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.042+0000 7fbfc1d9b640 1 -- 172.21.15.50:0/3758022978 >> v1:172.21.15.188:6789/0 conn(0x7fbfc40ce230 legacy=0x7fbfc40cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.043+0000 7fbfc1d9b640 1 -- 172.21.15.50:0/3758022978 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc40c4f50 con 0x7fbfc415a220 2024-04-02T16:22:28.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.044+0000 7fbfc1d9b640 1 --2- 172.21.15.50:0/3758022978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc4071770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4070db0 tx=0x7fbfb402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8470ddb88a17d43a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7fbfc159a640 1 -- 172.21.15.50:0/3758022978 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4034070 con 0x7fbfc415a220 2024-04-02T16:22:28.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7fbfc159a640 1 -- 172.21.15.50:0/3758022978 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4030650 con 0x7fbfc415a220 2024-04-02T16:22:28.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.046+0000 7fbfc159a640 1 -- 172.21.15.50:0/3758022978 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4030a20 con 0x7fbfc415a220 2024-04-02T16:22:28.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.046+0000 7fbfc359e640 1 -- 172.21.15.50:0/3758022978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 msgr2=0x7fbfc4071770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.046+0000 7fbfc359e640 1 --2- 172.21.15.50:0/3758022978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc4071770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4070db0 tx=0x7fbfb402ba20 comp rx=0 tx=0).stop 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.043+0000 7f8e79b82640 1 Processor -- start 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.043+0000 7f8e79b82640 1 -- start start 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79b82640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c053d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79b82640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8e6c054260 0x7f8e6c04f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79b82640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e6c052cf0 con 0x7f8e6c101010 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79b82640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f8e6c0341c0 con 0x7f8e6c054260 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79b82640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c038350 con 0x7f8e6c033920 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79381640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8e6c054260 0x7f8e6c04f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79381640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8e6c054260 0x7f8e6c04f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45220/0 (socket says 172.21.15.50:45220) 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.045+0000 7f8e79381640 1 -- 172.21.15.50:0/1622254187 learned_addr learned my addr 172.21.15.50:0/1622254187 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.046+0000 7f8e73fff640 1 --2- 172.21.15.50:0/1622254187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c053d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.047+0000 7f8e79381640 1 -- 172.21.15.50:0/1622254187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 msgr2=0x7f8e6c053d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.047+0000 7f8e79381640 1 --2- 172.21.15.50:0/1622254187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c053d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.047+0000 7f8e79381640 1 -- 172.21.15.50:0/1622254187 >> v1:172.21.15.188:6789/0 conn(0x7f8e6c101010 legacy=0x7f8e6c037c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.047+0000 7f8e79381640 1 -- 172.21.15.50:0/1622254187 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c052f60 con 0x7f8e6c054260 2024-04-02T16:22:28.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.046+0000 7f8e737fe640 1 -- 172.21.15.50:0/1622254187 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2808468483 0 0) 0x7f8e6c052cf0 con 0x7f8e6c101010 2024-04-02T16:22:28.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.048+0000 7fbfc359e640 1 -- 172.21.15.50:0/3758022978 shutdown_connections 2024-04-02T16:22:28.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.048+0000 7fbfc359e640 1 --2- 172.21.15.50:0/3758022978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc4071770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.048+0000 7fbfc359e640 1 --2- 172.21.15.50:0/3758022978 >> v2:172.21.15.188:3300/0 conn(0x7fbfc40d0770 0x7fbfc40cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.048+0000 7fbfc359e640 1 -- 172.21.15.50:0/3758022978 >> 172.21.15.50:0/3758022978 conn(0x7fbfc40cfc50 msgr2=0x7fbfc40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.048+0000 7f8e79381640 1 --2- 172.21.15.50:0/1622254187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c054260 0x7f8e6c04f290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8e7406b390 tx=0x7f8e7409a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa32c0ef2b728c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.049+0000 7f8e737fe640 1 -- 172.21.15.50:0/1622254187 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74072040 con 0x7f8e6c054260 2024-04-02T16:22:28.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.049+0000 7f8e737fe640 1 -- 172.21.15.50:0/1622254187 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7409e050 con 0x7f8e6c054260 2024-04-02T16:22:28.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.049+0000 7f8e737fe640 1 -- 172.21.15.50:0/1622254187 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e740a2900 con 0x7f8e6c054260 2024-04-02T16:22:28.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.049+0000 7fbfc359e640 1 -- 172.21.15.50:0/3758022978 shutdown_connections 2024-04-02T16:22:28.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.049+0000 7fbfc359e640 1 -- 172.21.15.50:0/3758022978 wait complete. 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.050+0000 7f8e79b82640 1 -- 172.21.15.50:0/1622254187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c054260 msgr2=0x7f8e6c04f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.050+0000 7f8e79b82640 1 --2- 172.21.15.50:0/1622254187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c054260 0x7f8e6c04f290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8e7406b390 tx=0x7f8e7409a7f0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.050+0000 7f8e79b82640 1 -- 172.21.15.50:0/1622254187 shutdown_connections 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.050+0000 7f8e79b82640 1 --2- 172.21.15.50:0/1622254187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c054260 0x7f8e6c04f290 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.050+0000 7f8e79b82640 1 --2- 172.21.15.50:0/1622254187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c053d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.050+0000 7f8e79b82640 1 -- 172.21.15.50:0/1622254187 >> 172.21.15.50:0/1622254187 conn(0x7f8e6c009fb0 msgr2=0x7f8e6c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.051+0000 7fbfc359e640 1 Processor -- start 2024-04-02T16:22:28.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.052+0000 7fbfc359e640 1 -- start start 2024-04-02T16:22:28.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.053+0000 7fbfc359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfc40ce230 0x7fbfc4155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.053+0000 7fbfc359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfc40d0770 0x7fbfc414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7fbfc359e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc406db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7fbfc1d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfc40d0770 0x7fbfc414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7fbfc1d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfc40d0770 0x7fbfc414b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57282/0 (socket says 172.21.15.50:57282) 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7fbfc1d9b640 1 -- 172.21.15.50:0/135690245 learned_addr learned my addr 172.21.15.50:0/135690245 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7fbfc359e640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc40cb4d0 con 0x7fbfc415a220 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7fbfc359e640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc40cb350 con 0x7fbfc40ce230 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7fbfc359e640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc40c5750 con 0x7fbfc40d0770 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.055+0000 7fbfc259c640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfc40ce230 0x7fbfc4155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.055+0000 7fbfc2d9d640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc406db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.051+0000 7f8e79b82640 1 -- 172.21.15.50:0/1622254187 shutdown_connections 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.051+0000 7f8e79b82640 1 -- 172.21.15.50:0/1622254187 wait complete. 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.052+0000 7f8e79b82640 1 Processor -- start 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.053+0000 7f8e79b82640 1 -- start start 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7f8e79b82640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c04d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7f8e79b82640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e6c054260 0x7f8e6c0f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7f8e78b80640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c04d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7f8e78b80640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c04d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57286/0 (socket says 172.21.15.50:57286) 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7f8e78b80640 1 -- 172.21.15.50:0/1985302205 learned_addr learned my addr 172.21.15.50:0/1985302205 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.054+0000 7f8e73fff640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e6c054260 0x7f8e6c0f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.055+0000 7f8e79b82640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c101010 0x7f8e6c1028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.055+0000 7f8e79b82640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c053800 con 0x7f8e6c101010 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.055+0000 7f8e79b82640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c053520 con 0x7f8e6c054260 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.055+0000 7f8e79b82640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c0546f0 con 0x7f8e6c033920 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7f8e79381640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c101010 0x7f8e6c1028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7f8e73fff640 1 -- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 msgr2=0x7f8e6c04d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7fbfc2d9d640 1 -- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfc40d0770 msgr2=0x7fbfc414b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7fbfc2d9d640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfc40d0770 0x7fbfc414b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7fbfc2d9d640 1 -- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfc40ce230 msgr2=0x7fbfc4155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7fbfc2d9d640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfc40ce230 0x7fbfc4155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7fbfc2d9d640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc414ce10 con 0x7fbfc415a220 2024-04-02T16:22:28.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7f8e73fff640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c04d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7f8e73fff640 1 -- 172.21.15.50:0/1985302205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c101010 msgr2=0x7f8e6c1028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7f8e73fff640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c101010 0x7f8e6c1028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7f8e73fff640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c04dde0 con 0x7f8e6c054260 2024-04-02T16:22:28.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.056+0000 7f8e79381640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c101010 0x7f8e6c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:28.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.057+0000 7f8e73fff640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e6c054260 0x7f8e6c0f8df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800fa70 tx=0x7f8e6800ff30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.057+0000 7fbfc2d9d640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc406db50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00c8c0 tx=0x7fbfbc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.057+0000 7f8e717fa640 1 -- 172.21.15.50:0/1985302205 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6801d070 con 0x7f8e6c054260 2024-04-02T16:22:28.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.057+0000 7f8e79b82640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c04e0c0 con 0x7f8e6c054260 2024-04-02T16:22:28.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.058+0000 7f8e79b82640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c04e680 con 0x7f8e6c054260 2024-04-02T16:22:28.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.058+0000 7fbfb2ffd640 1 -- 172.21.15.50:0/135690245 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc018070 con 0x7fbfc415a220 2024-04-02T16:22:28.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.058+0000 7fbfc1d9b640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfc40d0770 0x7fbfc414b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:28.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.058+0000 7fbfc359e640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc414d0f0 con 0x7fbfc415a220 2024-04-02T16:22:28.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.058+0000 7fbfc359e640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc406d630 con 0x7fbfc415a220 2024-04-02T16:22:28.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.058+0000 7f8e717fa640 1 -- 172.21.15.50:0/1985302205 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6800dda0 con 0x7f8e6c054260 2024-04-02T16:22:28.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.058+0000 7f8e717fa640 1 -- 172.21.15.50:0/1985302205 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68018660 con 0x7f8e6c054260 2024-04-02T16:22:28.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.061+0000 7f8e79b82640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c0089d0 con 0x7f8e6c054260 2024-04-02T16:22:28.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.061+0000 7fbfc359e640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc406d830 con 0x7fbfc415a220 2024-04-02T16:22:28.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.061+0000 7fbfb2ffd640 1 -- 172.21.15.50:0/135690245 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfbc00a640 con 0x7fbfc415a220 2024-04-02T16:22:28.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.061+0000 7fbfb2ffd640 1 -- 172.21.15.50:0/135690245 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00aa30 con 0x7fbfc415a220 2024-04-02T16:22:28.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.064+0000 7fbfb2ffd640 1 -- 172.21.15.50:0/135690245 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfbc01b070 con 0x7fbfc415a220 2024-04-02T16:22:28.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.064+0000 7fbfb2ffd640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbfac06cb00 0x7fbfac06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.065+0000 7fbfc259c640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbfac06cb00 0x7fbfac06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.065+0000 7fbfb2ffd640 1 -- 172.21.15.50:0/135690245 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbfbc08bb20 con 0x7fbfc415a220 2024-04-02T16:22:28.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.066+0000 7fbfc259c640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbfac06cb00 0x7fbfac06efc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb80090e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.062+0000 7f8e717fa640 1 -- 172.21.15.50:0/1985302205 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e68020070 con 0x7f8e6c054260 2024-04-02T16:22:28.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.066+0000 7f8e717fa640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8e5806c9c0 0x7f8e5806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.066+0000 7f8e717fa640 1 -- 172.21.15.50:0/1985302205 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8e6808c350 con 0x7f8e6c054260 2024-04-02T16:22:28.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.067+0000 7f8e78b80640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8e5806c9c0 0x7f8e5806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.067+0000 7f8e717fa640 1 -- 172.21.15.50:0/1985302205 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8e68059e60 con 0x7f8e6c054260 2024-04-02T16:22:28.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.072+0000 7fbfb2ffd640 1 -- 172.21.15.50:0/135690245 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbfc406d830 con 0x7fbfc415a220 2024-04-02T16:22:28.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.077+0000 7f8e78b80640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8e5806c9c0 0x7f8e5806ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c100720 tx=0x7f8e60009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.098+0000 7fe88bd15640 1 Processor -- start 2024-04-02T16:22:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.099+0000 7fa1ab0ba640 1 Processor -- start 2024-04-02T16:22:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.099+0000 7fa1ab0ba640 1 -- start start 2024-04-02T16:22:28.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.099+0000 7fe88bd15640 1 -- start start 2024-04-02T16:22:28.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.099+0000 7f148f31a640 1 Processor -- start 2024-04-02T16:22:28.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.099+0000 7f148f31a640 1 -- start start 2024-04-02T16:22:28.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7f155bfff640 1 Processor -- start 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7fe88bd15640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe88415e5d0 0x7fe8840cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7fe88bd15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840cf210 0x7fe8840d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7fe88bd15640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8840d4b40 con 0x7fe8840d46b0 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7fe88bd15640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe88415e9b0 con 0x7fe88415e5d0 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7fe88bd15640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe8840d03c0 con 0x7fe8840cf210 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7fe88a2b2640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe8840d46b0 0x7fe884162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55078/0 (socket says 172.21.15.50:55078) 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fe88a2b2640 1 -- 172.21.15.50:0/967306603 learned_addr learned my addr 172.21.15.50:0/967306603 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:28.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fe8892b0640 1 --2- 172.21.15.50:0/967306603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840cf210 0x7fe8840d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fa1a8e56640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa1a40cdee0 0x7fa1a40d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55080/0 (socket says 172.21.15.50:55080) 2024-04-02T16:22:28.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fa1a8e56640 1 -- 172.21.15.50:0/2993505400 learned_addr learned my addr 172.21.15.50:0/2993505400 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:28.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fe8892b0640 1 -- 172.21.15.50:0/967306603 >> v2:172.21.15.188:3300/0 conn(0x7fe88415e5d0 msgr2=0x7fe8840cecd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T16:22:28.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fe8892b0640 1 --2- 172.21.15.50:0/967306603 >> v2:172.21.15.188:3300/0 conn(0x7fe88415e5d0 0x7fe8840cecd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fe8892b0640 1 -- 172.21.15.50:0/967306603 >> v1:172.21.15.50:6789/0 conn(0x7fe8840d46b0 legacy=0x7fe884162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7fe8892b0640 1 -- 172.21.15.50:0/967306603 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8840cf7b0 con 0x7fe8840cf210 2024-04-02T16:22:28.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f148f31a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1480050160 0x7f1480038040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f148f31a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f148e318640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f14800df1e0 0x7f148004fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55092/0 (socket says 172.21.15.50:55092) 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f148e318640 1 -- 172.21.15.50:0/2152985066 learned_addr learned my addr 172.21.15.50:0/2152985066 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1480054480 con 0x7f14800df1e0 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f14800077a0 con 0x7f1480050160 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.100+0000 7f155bfff640 1 -- start start 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f155bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f155c0c42b0 0x7f155c072db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f155bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f155c166ea0 0x7f155c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f155bfff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155c0cbeb0 con 0x7f155c06b520 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f155bfff640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f155c0c4690 con 0x7f155c0c42b0 2024-04-02T16:22:28.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f155bfff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f155c0722d0 con 0x7f155c166ea0 2024-04-02T16:22:28.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f5edcf6f640 1 Processor -- start 2024-04-02T16:22:28.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.101+0000 7f5edcf6f640 1 -- start start 2024-04-02T16:22:28.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f148eb19640 1 --2- 172.21.15.50:0/2152985066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2993505400 >> v2:172.21.15.188:3300/0 conn(0x7fa1a415b8a0 0x7fa1a40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f7238b11640 1 Processor -- start 2024-04-02T16:22:28.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f155b7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f155c06b520 0x7f155c06b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:44082/0 (socket says 172.21.15.50:44082) 2024-04-02T16:22:28.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f155b7fe640 1 -- 172.21.15.50:0/2432391435 learned_addr learned my addr 172.21.15.50:0/2432391435 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:28.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f7238b11640 1 -- start start 2024-04-02T16:22:28.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f155affd640 1 --2- 172.21.15.50:0/2432391435 >> v2:172.21.15.50:3300/0 conn(0x7f155c0c42b0 0x7f155c072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f148db17640 1 --2- 172.21.15.50:0/2152985066 >> v2:172.21.15.50:3300/0 conn(0x7f1480050160 0x7f1480038040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f155a7fc640 1 --2- 172.21.15.50:0/2432391435 >> v2:172.21.15.188:3300/0 conn(0x7f155c166ea0 0x7f155c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a40d1090 con 0x7fa1a40cdee0 2024-04-02T16:22:28.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a40afc10 con 0x7fa1a40b09d0 2024-04-02T16:22:28.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a415c050 con 0x7fa1a415b8a0 2024-04-02T16:22:28.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508447525 0 0) 0x7f155c0cbeb0 con 0x7f155c06b520 2024-04-02T16:22:28.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1544003520 con 0x7f155c06b520 2024-04-02T16:22:28.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.102+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f14800541a0 con 0x7f1480053c30 2024-04-02T16:22:28.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5edcf6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed80c3850 0x7f5ed80cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5edcf6f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5ed806e670 0x7f5ed806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5edcf6f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed80cb8e0 con 0x7f5ed8162a50 2024-04-02T16:22:28.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5edcf6f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed80cc1d0 con 0x7f5ed806e670 2024-04-02T16:22:28.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5edcf6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed80cbb80 con 0x7f5ed80c3850 2024-04-02T16:22:28.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7fa1a3fff640 1 --2- 172.21.15.50:0/2993505400 >> v2:172.21.15.188:3300/0 conn(0x7fa1a415b8a0 0x7fa1a40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7fe8892b0640 1 --2- 172.21.15.50:0/967306603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840cf210 0x7fe8840d4170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009d90 tx=0x7fe87c02bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede328800168437 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2603680372 0 0) 0x7f1544003520 con 0x7f155c06b520 2024-04-02T16:22:28.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 >> v2:172.21.15.188:3300/0 conn(0x7f155c166ea0 msgr2=0x7f155c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 --2- 172.21.15.50:0/2432391435 >> v2:172.21.15.188:3300/0 conn(0x7f155c166ea0 0x7f155c0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 >> v2:172.21.15.50:3300/0 conn(0x7f155c0c42b0 msgr2=0x7f155c072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 --2- 172.21.15.50:0/2432391435 >> v2:172.21.15.50:3300/0 conn(0x7f155c0c42b0 0x7f155c072db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c0725b0 con 0x7f155c06b520 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f15540019d0 con 0x7f155c06b520 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 >> v1:172.21.15.188:6789/0 conn(0x7f155c06b520 legacy=0x7f155c06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f7238b11640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72340b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f7238b11640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7234161b40 0x7f72340cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f7238b11640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72340cf970 con 0x7f7234161760 2024-04-02T16:22:28.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f7238b11640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f72340d1cc0 con 0x7f7234161b40 2024-04-02T16:22:28.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15440039f0 0x7f1544003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1544004800 0x7f1544004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f154400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f155c0c4690 con 0x7f15440051c0 2024-04-02T16:22:28.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f155c0722d0 con 0x7f15440039f0 2024-04-02T16:22:28.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f154400c0c0 con 0x7f1544004800 2024-04-02T16:22:28.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f7238b11640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f72340d1fa0 con 0x7f72340d1160 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f148d316640 1 -- 172.21.15.50:0/2152985066 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104075439 0 0) 0x7f1480054480 con 0x7f14800df1e0 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f148d316640 1 -- 172.21.15.50:0/2152985066 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1474005730 con 0x7f14800df1e0 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f148eb19640 1 -- 172.21.15.50:0/2152985066 >> v2:172.21.15.50:3300/0 conn(0x7f1480050160 msgr2=0x7f1480038040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f148eb19640 1 --2- 172.21.15.50:0/2152985066 >> v2:172.21.15.50:3300/0 conn(0x7f1480050160 0x7f1480038040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f148eb19640 1 -- 172.21.15.50:0/2152985066 >> v1:172.21.15.50:6789/0 conn(0x7f14800df1e0 legacy=0x7f148004fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f148eb19640 1 -- 172.21.15.50:0/2152985066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1480005ef0 con 0x7f1480053c30 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f148db17640 1 --2- 172.21.15.50:0/2152985066 >> v2:172.21.15.50:3300/0 conn(0x7f1480050160 0x7f1480038040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2554161931 0 0) 0x7fa1a40afc10 con 0x7fa1a40b09d0 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa190003520 con 0x7fa1a40b09d0 2024-04-02T16:22:28.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695340063 0 0) 0x7fa1a40d1090 con 0x7fa1a40cdee0 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1a40afc10 con 0x7fa1a40cdee0 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2444089385 0 0) 0x7fa190003520 con 0x7fa1a40b09d0 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 >> v2:172.21.15.188:3300/0 conn(0x7fa1a415b8a0 msgr2=0x7fa1a40ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 --2- 172.21.15.50:0/2993505400 >> v2:172.21.15.188:3300/0 conn(0x7fa1a415b8a0 0x7fa1a40ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 >> v1:172.21.15.50:6789/0 conn(0x7fa1a40cdee0 legacy=0x7fa1a40d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40afef0 con 0x7fa1a40b09d0 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fa194001920 con 0x7fa1a40b09d0 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 >> v1:172.21.15.188:6789/0 conn(0x7fa1a40b09d0 legacy=0x7fa1a40b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa190005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1900037a0 0x7fa190003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa190004160 0x7fa19000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa19000c000 con 0x7fa190004c40 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa19000c1b0 con 0x7fa1900037a0 2024-04-02T16:22:28.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa19000c3b0 con 0x7fa190004160 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a8e56640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa190005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7fe888aaf640 1 -- 172.21.15.50:0/967306603 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c039070 con 0x7fe8840cf210 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fe888aaf640 1 -- 172.21.15.50:0/967306603 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87c02cc70 con 0x7fe8840cf210 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7fe888aaf640 1 -- 172.21.15.50:0/967306603 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c034660 con 0x7fe8840cf210 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed6ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed80c3850 0x7f5ed80cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed6ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed80c3850 0x7f5ed80cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57300/0 (socket says 172.21.15.50:57300) 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed6ffd640 1 -- 172.21.15.50:0/2536685678 learned_addr learned my addr 172.21.15.50:0/2536685678 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed7fff640 1 --2- 172.21.15.50:0/2536685678 >> v2:172.21.15.50:3300/0 conn(0x7f5ed806e670 0x7f5ed806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed6ffd640 1 -- 172.21.15.50:0/2536685678 >> v2:172.21.15.50:3300/0 conn(0x7f5ed806e670 msgr2=0x7f5ed806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed6ffd640 1 --2- 172.21.15.50:0/2536685678 >> v2:172.21.15.50:3300/0 conn(0x7f5ed806e670 0x7f5ed806eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed6ffd640 1 -- 172.21.15.50:0/2536685678 >> v1:172.21.15.188:6789/0 conn(0x7f5ed8162a50 legacy=0x7f5ed80c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f5ed6ffd640 1 -- 172.21.15.50:0/2536685678 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed80cbe60 con 0x7f5ed80c3850 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f7232d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7234161b40 0x7f72340cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f723259c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7234161760 0x7f72340d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:44110/0 (socket says 172.21.15.50:44110) 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f723259c640 1 -- 172.21.15.50:0/498194809 learned_addr learned my addr 172.21.15.50:0/498194809 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f7232d9d640 1 --2- 172.21.15.50:0/498194809 >> v2:172.21.15.188:3300/0 conn(0x7f7234161b40 0x7f72340cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42674/0 (socket says 172.21.15.50:42674) 2024-04-02T16:22:28.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f722bfff640 1 --2- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72340b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.103+0000 7f155a7fc640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1544004800 0x7f1544004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f155b7fe640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f154400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f155b7fe640 1 -- 172.21.15.50:0/2432391435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1544004800 msgr2=0x7f1544004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f148eb19640 1 --2- 172.21.15.50:0/2152985066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800069d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1488051160 tx=0x7f14880a6d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd8b17f037abfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f155b7fe640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1544004800 0x7f1544004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f155b7fe640 1 -- 172.21.15.50:0/2432391435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15440039f0 msgr2=0x7f1544003e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T16:22:28.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f155b7fe640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15440039f0 0x7f1544003e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f155b7fe640 1 -- 172.21.15.50:0/2432391435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c0725b0 con 0x7f15440051c0 2024-04-02T16:22:28.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f7232d9d640 1 -- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 msgr2=0x7f72340b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f7232d9d640 1 --2- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72340b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f7232d9d640 1 -- 172.21.15.50:0/498194809 >> v1:172.21.15.188:6789/0 conn(0x7f7234161760 legacy=0x7f72340d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.104+0000 7f7232d9d640 1 -- 172.21.15.50:0/498194809 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340cfc50 con 0x7f7234161b40 2024-04-02T16:22:28.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f148d316640 1 -- 172.21.15.50:0/2152985066 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880b4070 con 0x7f1480053c30 2024-04-02T16:22:28.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f148d316640 1 -- 172.21.15.50:0/2152985066 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14880a7c00 con 0x7f1480053c30 2024-04-02T16:22:28.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f148d316640 1 -- 172.21.15.50:0/2152985066 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880af660 con 0x7f1480053c30 2024-04-02T16:22:28.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fe88bd15640 1 -- 172.21.15.50:0/967306603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840cf210 msgr2=0x7fe8840d4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fe88bd15640 1 --2- 172.21.15.50:0/967306603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840cf210 0x7fe8840d4170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009d90 tx=0x7fe87c02bec0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a3fff640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1900037a0 0x7fa190003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a9657640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa190004160 0x7fa19000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a8e56640 1 -- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa190004160 msgr2=0x7fa19000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a8e56640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa190004160 0x7fa19000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f7232d9d640 1 --2- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161b40 0x7f72340cec90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f722c009d90 tx=0x7f722c02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd49e50007a1c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a8e56640 1 -- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1900037a0 msgr2=0x7fa190003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a8e56640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1900037a0 0x7fa190003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 msgr2=0x7f14800069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f148f31a640 1 --2- 172.21.15.50:0/2152985066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800069d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1488051160 tx=0x7f14880a6d50 comp rx=0 tx=0).stop 2024-04-02T16:22:28.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a8e56640 1 -- 172.21.15.50:0/2993505400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa190003520 con 0x7fa190004c40 2024-04-02T16:22:28.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a3fff640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1900037a0 0x7fa190003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:28.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a9657640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa190004160 0x7fa19000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:28.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fa1a9657640 1 -- 172.21.15.50:0/2993505400 reap_dead start 2024-04-02T16:22:28.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fe88bd15640 1 -- 172.21.15.50:0/967306603 shutdown_connections 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fe88bd15640 1 --2- 172.21.15.50:0/967306603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840cf210 0x7fe8840d4170 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fe88bd15640 1 --2- 172.21.15.50:0/967306603 >> v2:172.21.15.188:3300/0 conn(0x7fe88415e5d0 0x7fe8840cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7fe88bd15640 1 -- 172.21.15.50:0/967306603 >> 172.21.15.50:0/967306603 conn(0x7fe88408dce0 msgr2=0x7fe88409d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7fe88bd15640 1 -- 172.21.15.50:0/967306603 shutdown_connections 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7fe88bd15640 1 -- 172.21.15.50:0/967306603 wait complete. 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f7231d9b640 1 -- 172.21.15.50:0/498194809 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c039070 con 0x7f7234161b40 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f7231d9b640 1 -- 172.21.15.50:0/498194809 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c02cc00 con 0x7f7234161b40 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f155b7fe640 1 -- 172.21.15.50:0/2432391435 reap_dead start 2024-04-02T16:22:28.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f155b7fe640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f154400b9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f155400a990 tx=0x7f1544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb949fb9e6bb5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f7231d9b640 1 -- 172.21.15.50:0/498194809 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c034440 con 0x7f7234161b40 2024-04-02T16:22:28.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400a560 con 0x7f15440051c0 2024-04-02T16:22:28.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1554034650 con 0x7f15440051c0 2024-04-02T16:22:28.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f1559ffb640 1 -- 172.21.15.50:0/2432391435 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554034a40 con 0x7f15440051c0 2024-04-02T16:22:28.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f7238b11640 1 -- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161b40 msgr2=0x7f72340cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 shutdown_connections 2024-04-02T16:22:28.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f148f31a640 1 --2- 172.21.15.50:0/2152985066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800069d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f148f31a640 1 --2- 172.21.15.50:0/2152985066 >> v2:172.21.15.50:3300/0 conn(0x7f1480050160 0x7f1480038040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7fa1a8e56640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa190005040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa19c001780 tx=0x7fa190005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f04d5d5d0d3a983 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 >> 172.21.15.50:0/2152985066 conn(0x7f1480009f70 msgr2=0x7f148000a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.106+0000 7f7238b11640 1 --2- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161b40 0x7f72340cec90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f722c009d90 tx=0x7f722c02be50 comp rx=0 tx=0).stop 2024-04-02T16:22:28.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.105+0000 7f5ed6ffd640 1 --2- 172.21.15.50:0/2536685678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed80c3850 0x7f5ed80cc660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009e50 tx=0x7f5ed002bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ac6e88aa6a22a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 shutdown_connections 2024-04-02T16:22:28.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f148f31a640 1 -- 172.21.15.50:0/2152985066 wait complete. 2024-04-02T16:22:28.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f7238b11640 1 -- 172.21.15.50:0/498194809 shutdown_connections 2024-04-02T16:22:28.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f7238b11640 1 --2- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161b40 0x7f72340cec90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c037070 con 0x7fa190004c40 2024-04-02T16:22:28.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f7238b11640 1 --2- 172.21.15.50:0/498194809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72340b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f5ed67fc640 1 -- 172.21.15.50:0/2536685678 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed002cb20 con 0x7f5ed80c3850 2024-04-02T16:22:28.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f5ed67fc640 1 -- 172.21.15.50:0/2536685678 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed002ccc0 con 0x7f5ed80c3850 2024-04-02T16:22:28.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c02d040 con 0x7fa190004c40 2024-04-02T16:22:28.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7fa1a37fe640 1 -- 172.21.15.50:0/2993505400 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00c040 con 0x7fa190004c40 2024-04-02T16:22:28.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f7238b11640 1 -- 172.21.15.50:0/498194809 >> 172.21.15.50:0/498194809 conn(0x7f723409d460 msgr2=0x7f72340d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f7238b11640 1 -- 172.21.15.50:0/498194809 shutdown_connections 2024-04-02T16:22:28.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f7238b11640 1 -- 172.21.15.50:0/498194809 wait complete. 2024-04-02T16:22:28.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 -- 172.21.15.50:0/2536685678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed80c3850 msgr2=0x7f5ed80cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/2536685678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed80c3850 0x7f5ed80cc660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009e50 tx=0x7f5ed002bf80 comp rx=0 tx=0).stop 2024-04-02T16:22:28.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 -- 172.21.15.50:0/2536685678 shutdown_connections 2024-04-02T16:22:28.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/2536685678 >> v2:172.21.15.50:3300/0 conn(0x7f5ed806e670 0x7f5ed806eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/2536685678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed80c3850 0x7f5ed80cc660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 -- 172.21.15.50:0/2536685678 >> 172.21.15.50:0/2536685678 conn(0x7f5ed80d0170 msgr2=0x7f5ed80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 msgr2=0x7fa190005040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa190005040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa19c001780 tx=0x7fa190005830 comp rx=0 tx=0).stop 2024-04-02T16:22:28.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 -- 172.21.15.50:0/2536685678 shutdown_connections 2024-04-02T16:22:28.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7f5edcf6f640 1 -- 172.21.15.50:0/2536685678 wait complete. 2024-04-02T16:22:28.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fe88bd15640 1 Processor -- start 2024-04-02T16:22:28.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fe88bd15640 1 -- start start 2024-04-02T16:22:28.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 shutdown_connections 2024-04-02T16:22:28.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2993505400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa190005040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 >> 172.21.15.50:0/2993505400 conn(0x7fa1a4072b40 msgr2=0x7fa1a4072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 shutdown_connections 2024-04-02T16:22:28.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.108+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2993505400 wait complete. 2024-04-02T16:22:28.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f155bfff640 1 -- 172.21.15.50:0/2432391435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 msgr2=0x7f154400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f155bfff640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f154400b9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f155400a990 tx=0x7f1544005830 comp rx=0 tx=0).stop 2024-04-02T16:22:28.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f155bfff640 1 -- 172.21.15.50:0/2432391435 shutdown_connections 2024-04-02T16:22:28.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f155bfff640 1 --2- 172.21.15.50:0/2432391435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f154400b9c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.107+0000 7f155bfff640 1 -- 172.21.15.50:0/2432391435 >> 172.21.15.50:0/2432391435 conn(0x7f155c0d0de0 msgr2=0x7f155c0d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.109+0000 7f155bfff640 1 -- 172.21.15.50:0/2432391435 shutdown_connections 2024-04-02T16:22:28.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.109+0000 7f155bfff640 1 -- 172.21.15.50:0/2432391435 wait complete. 2024-04-02T16:22:28.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.109+0000 7f148f31a640 1 Processor -- start 2024-04-02T16:22:28.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe88bd15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8840cf210 0x7fe8840b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7f7238b11640 1 Processor -- start 2024-04-02T16:22:28.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe88bd15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840d46b0 0x7fe88415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7f5edcf6f640 1 Processor -- start 2024-04-02T16:22:28.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe889ab1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8840cf210 0x7fe8840b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe889ab1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8840cf210 0x7fe8840b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42698/0 (socket says 172.21.15.50:42698) 2024-04-02T16:22:28.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe889ab1640 1 -- 172.21.15.50:0/2382718043 learned_addr learned my addr 172.21.15.50:0/2382718043 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe8892b0640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840d46b0 0x7fe88415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe88bd15640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88415e5d0 0x7fe8840b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8840d0050 con 0x7fe88415e5d0 2024-04-02T16:22:28.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe8840cfd70 con 0x7fe8840cf210 2024-04-02T16:22:28.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe8840b5820 con 0x7fe8840d46b0 2024-04-02T16:22:28.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f148f31a640 1 -- start start 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7f7238b11640 1 -- start start 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f7238b11640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72341514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f155bfff640 1 Processor -- start 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7fe88a2b2640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88415e5d0 0x7fe8840b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7fe88a2b2640 1 -- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840d46b0 msgr2=0x7fe88415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7fe88a2b2640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840d46b0 0x7fe88415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7fe88a2b2640 1 -- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8840cf210 msgr2=0x7fe8840b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7fe88a2b2640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8840cf210 0x7fe8840b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7fe88a2b2640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8840b13d0 con 0x7fe88415e5d0 2024-04-02T16:22:28.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f7238b11640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161760 0x7f72340c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.110+0000 7f5edcf6f640 1 -- start start 2024-04-02T16:22:28.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f155bfff640 1 -- start start 2024-04-02T16:22:28.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f723259c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72341514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f723259c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72341514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57340/0 (socket says 172.21.15.50:57340) 2024-04-02T16:22:28.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f723259c640 1 -- 172.21.15.50:0/3514799601 learned_addr learned my addr 172.21.15.50:0/3514799601 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7fe88a2b2640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88415e5d0 0x7fe8840b3c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c390 tx=0x7fe88000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f7238b11640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7234161b40 0x7f723415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f5edcf6f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ed806e670 0x7f5ed8165980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.111+0000 7f722bfff640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161760 0x7f72340c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f72340cf1d0 con 0x7f7234161b40 2024-04-02T16:22:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f7232d9d640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7234161b40 0x7f723415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5ed77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ed806e670 0x7f5ed8165980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5ed77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ed806e670 0x7f5ed8165980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45306/0 (socket says 172.21.15.50:45306) 2024-04-02T16:22:28.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7fe87a7fc640 1 -- 172.21.15.50:0/2382718043 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880017070 con 0x7fe88415e5d0 2024-04-02T16:22:28.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7fe87a7fc640 1 -- 172.21.15.50:0/2382718043 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe880005ce0 con 0x7fe88415e5d0 2024-04-02T16:22:28.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7fe87a7fc640 1 -- 172.21.15.50:0/2382718043 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8800072d0 con 0x7fe88415e5d0 2024-04-02T16:22:28.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5ed77fe640 1 -- 172.21.15.50:0/3744092765 learned_addr learned my addr 172.21.15.50:0/3744092765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148f31a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1480050160 0x7f14800db440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148f31a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800dd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148f31a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14800df1e0 0x7f14800ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148f31a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14800378d0 con 0x7f14800df1e0 2024-04-02T16:22:28.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148f31a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14800064b0 con 0x7f1480050160 2024-04-02T16:22:28.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148f31a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1480037a50 con 0x7f1480053c30 2024-04-02T16:22:28.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed8162a50 0x7f5ed80c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed80c9e70 0x7f5ed80ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7fa1ab0ba640 1 Processor -- start 2024-04-02T16:22:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f72340b52b0 con 0x7f7234161760 2024-04-02T16:22:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed806f270 con 0x7f5ed806e670 2024-04-02T16:22:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148eb19640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14800df1e0 0x7f14800ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148eb19640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14800df1e0 0x7f14800ddea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45326/0 (socket says 172.21.15.50:45326) 2024-04-02T16:22:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f148eb19640 1 -- 172.21.15.50:0/4123107748 learned_addr learned my addr 172.21.15.50:0/4123107748 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5ed6ffd640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed8162a50 0x7f5ed80c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7fa1ab0ba640 1 -- start start 2024-04-02T16:22:28.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed806f0f0 con 0x7f5ed80c9e70 2024-04-02T16:22:28.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5ed7fff640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed80c9e70 0x7f5ed80ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed80ce070 con 0x7f5ed8162a50 2024-04-02T16:22:28.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f72340cf350 con 0x7f72340d1160 2024-04-02T16:22:28.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f148eb19640 1 -- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 msgr2=0x7f14800dd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f148eb19640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f148eb19640 1 -- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1480050160 msgr2=0x7f14800db440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f148e318640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1480050160 0x7f14800db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f155c0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c156700 0x7f155c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f155c0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f5ed77fe640 1 -- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed8162a50 msgr2=0x7f5ed80c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f155c0c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45338/0 (socket says 172.21.15.50:45338) 2024-04-02T16:22:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155affd640 1 -- 172.21.15.50:0/3513188076 learned_addr learned my addr 172.21.15.50:0/3513188076 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155bfff640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c0b5b00 0x7f155c0b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f5ed77fe640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed8162a50 0x7f5ed80c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f5ed77fe640 1 -- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed80c9e70 msgr2=0x7f5ed80ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f5ed77fe640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed80c9e70 0x7f5ed80ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f5ed77fe640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed80c8df0 con 0x7f5ed806e670 2024-04-02T16:22:28.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155a7fc640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c156700 0x7f155c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155bfff640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f155c072db0 con 0x7f15440051c0 2024-04-02T16:22:28.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155bfff640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f155c0cc190 con 0x7f155c0b5b00 2024-04-02T16:22:28.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f155bfff640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f155c072f30 con 0x7f155c156700 2024-04-02T16:22:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f7232d9d640 1 -- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 msgr2=0x7f72341514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f7232d9d640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72341514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f7232d9d640 1 -- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161760 msgr2=0x7f72340c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f7232d9d640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161760 0x7f72340c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f7232d9d640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340c93b0 con 0x7f7234161b40 2024-04-02T16:22:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.112+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8840b16b0 con 0x7fe88415e5d0 2024-04-02T16:22:28.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8840b1c70 con 0x7fe88415e5d0 2024-04-02T16:22:28.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1ab0ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa1a40c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f148eb19640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1480050160 0x7f14800db440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f7232d9d640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7234161b40 0x7f723415abd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f722c033000 tx=0x7f722c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.113+0000 7f148eb19640 1 -- 172.21.15.50:0/4123107748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148017d290 con 0x7f14800df1e0 2024-04-02T16:22:28.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1ab0ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a41f32c0 0x7fa1a40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f155b7fe640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c0b5b00 0x7f155c0b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f148e318640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1480050160 0x7f14800db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:28.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1ab0ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a406d8c0 0x7fa1a40c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1a8e56640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa1a40c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1ab0ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41584d0 con 0x7fa190004c40 2024-04-02T16:22:28.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a41f32c0 0x7fa1a40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a41f32c0 0x7fa1a40c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:42742/0 (socket says 172.21.15.50:42742) 2024-04-02T16:22:28.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1a3fff640 1 -- 172.21.15.50:0/2560461846 learned_addr learned my addr 172.21.15.50:0/2560461846 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f155a7fc640 1 -- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c0b5b00 msgr2=0x7f155c0b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f155a7fc640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c0b5b00 0x7f155c0b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f155a7fc640 1 -- 172.21.15.50:0/3513188076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 msgr2=0x7f155c0c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f155a7fc640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f155c0c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7f155a7fc640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c15fc50 con 0x7f155c156700 2024-04-02T16:22:28.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe884075760 con 0x7fe88415e5d0 2024-04-02T16:22:28.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f155b7fe640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c0b5b00 0x7f155c0b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:28.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f5ed77fe640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ed806e670 0x7f5ed8165980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c400 tx=0x7f5ecc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1a9657640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a406d8c0 0x7fa1a40c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a40b04b0 con 0x7fa1a41f32c0 2024-04-02T16:22:28.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.114+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a4158650 con 0x7fa1a406d8c0 2024-04-02T16:22:28.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f155a7fc640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c156700 0x7f155c0c3790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f155c163d50 tx=0x7f154c00f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f155affd640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f155c0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:28.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f5ebffff640 1 -- 172.21.15.50:0/3744092765 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc017070 con 0x7f5ed806e670 2024-04-02T16:22:28.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7fa1a3fff640 1 -- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a406d8c0 msgr2=0x7fa1a40c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7fa1a3fff640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a406d8c0 0x7fa1a40c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7fa1a3fff640 1 -- 172.21.15.50:0/2560461846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 msgr2=0x7fa1a40c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7fa1a3fff640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa1a40c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7fa1a3fff640 1 -- 172.21.15.50:0/2560461846 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a406e1c0 con 0x7fa1a41f32c0 2024-04-02T16:22:28.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f148eb19640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14800df1e0 0x7f14800ddea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f148806a330 tx=0x7f148806b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7fa1a3fff640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a41f32c0 0x7fa1a40c53e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa18c011cd0 tx=0x7fa18c00b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f1543fff640 1 -- 172.21.15.50:0/3513188076 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c004440 con 0x7f155c156700 2024-04-02T16:22:28.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f1543fff640 1 -- 172.21.15.50:0/3513188076 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c01a070 con 0x7f155c156700 2024-04-02T16:22:28.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f1543fff640 1 -- 172.21.15.50:0/3513188076 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c003810 con 0x7f155c156700 2024-04-02T16:22:28.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7f155bfff640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c15ff30 con 0x7f155c156700 2024-04-02T16:22:28.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed80c90b0 con 0x7f5ed806e670 2024-04-02T16:22:28.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed80c9670 con 0x7f5ed806e670 2024-04-02T16:22:28.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7f155bfff640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c1604f0 con 0x7f155c156700 2024-04-02T16:22:28.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f5ebffff640 1 -- 172.21.15.50:0/3744092765 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ecc0052d0 con 0x7f5ed806e670 2024-04-02T16:22:28.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.115+0000 7f5ebffff640 1 -- 172.21.15.50:0/3744092765 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc003830 con 0x7f5ed806e670 2024-04-02T16:22:28.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7fa1a17fa640 1 -- 172.21.15.50:0/2560461846 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c01f070 con 0x7fa1a41f32c0 2024-04-02T16:22:28.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7fa1a17fa640 1 -- 172.21.15.50:0/2560461846 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18c01a410 con 0x7fa1a41f32c0 2024-04-02T16:22:28.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7fa1a17fa640 1 -- 172.21.15.50:0/2560461846 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c01a7c0 con 0x7fa1a41f32c0 2024-04-02T16:22:28.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7f147effd640 1 -- 172.21.15.50:0/4123107748 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880b4050 con 0x7f14800df1e0 2024-04-02T16:22:28.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7f147effd640 1 -- 172.21.15.50:0/4123107748 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14880b5040 con 0x7f14800df1e0 2024-04-02T16:22:28.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.117+0000 7f147effd640 1 -- 172.21.15.50:0/4123107748 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880b9810 con 0x7f14800df1e0 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7f722affd640 1 -- 172.21.15.50:0/3514799601 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c043020 con 0x7f7234161b40 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7f722affd640 1 -- 172.21.15.50:0/3514799601 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c03a040 con 0x7f7234161b40 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.117+0000 7f722affd640 1 -- 172.21.15.50:0/3514799601 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c039070 con 0x7f7234161b40 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.118+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72340c9690 con 0x7f7234161b40 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.118+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72340c9c50 con 0x7f7234161b40 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.116+0000 7fe87a7fc640 1 -- 172.21.15.50:0/2382718043 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe88001a070 con 0x7fe88415e5d0 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.117+0000 7fe87a7fc640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe86806c9f0 0x7fe86806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.117+0000 7fe87a7fc640 1 -- 172.21.15.50:0/2382718043 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe88008aff0 con 0x7fe88415e5d0 2024-04-02T16:22:28.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.118+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1534005290 con 0x7f155c156700 2024-04-02T16:22:28.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.118+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a40c4e90 con 0x7fa1a41f32c0 2024-04-02T16:22:28.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.117+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148017d570 con 0x7f14800df1e0 2024-04-02T16:22:28.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.118+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a40cbe10 con 0x7fa1a41f32c0 2024-04-02T16:22:28.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.118+0000 7f5ebffff640 1 -- 172.21.15.50:0/3744092765 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ecc01a070 con 0x7f5ed806e670 2024-04-02T16:22:28.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.119+0000 7f5ebffff640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5ec406c8f0 0x7f5ec406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.119+0000 7f5ed6ffd640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5ec406c8f0 0x7f5ec406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.119+0000 7f1543fff640 1 -- 172.21.15.50:0/3513188076 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f154c012070 con 0x7f155c156700 2024-04-02T16:22:28.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.117+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148017db30 con 0x7f14800df1e0 2024-04-02T16:22:28.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.118+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1480005350 con 0x7f14800df1e0 2024-04-02T16:22:28.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.119+0000 7f1543fff640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f154806c940 0x7f154806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.119+0000 7f1543fff640 1 -- 172.21.15.50:0/3513188076 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f154c08ba80 con 0x7f155c156700 2024-04-02T16:22:28.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72340756f0 con 0x7f7234161b40 2024-04-02T16:22:28.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7f155affd640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f154806c940 0x7f154806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7fe889ab1640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe86806c9f0 0x7fe86806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.119+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb8005290 con 0x7f5ed806e670 2024-04-02T16:22:28.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4075760 con 0x7fa1a41f32c0 2024-04-02T16:22:28.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7f5ed6ffd640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5ec406c8f0 0x7f5ec406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0035000 tx=0x7f5ed0030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7f155affd640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f154806c940 0x7f154806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1550009fb0 tx=0x7f1550009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7fa1a17fa640 1 -- 172.21.15.50:0/2560461846 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa18c022070 con 0x7fa1a41f32c0 2024-04-02T16:22:28.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.120+0000 7fe889ab1640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe86806c9f0 0x7fe86806eeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe874006fd0 tx=0x7fe874008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.121+0000 7fa1a17fa640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa18406c970 0x7fa18406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.121+0000 7fa1a17fa640 1 -- 172.21.15.50:0/2560461846 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa18c08f9e0 con 0x7fa1a41f32c0 2024-04-02T16:22:28.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.121+0000 7fa1a8e56640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa18406c970 0x7fa18406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.121+0000 7fa1a8e56640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa18406c970 0x7fa18406ee30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa19c007fd0 tx=0x7fa19c02f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.123+0000 7f5ebffff640 1 -- 172.21.15.50:0/3744092765 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5ecc056c90 con 0x7f5ed806e670 2024-04-02T16:22:28.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.124+0000 7fe87a7fc640 1 -- 172.21.15.50:0/2382718043 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe880058b00 con 0x7fe88415e5d0 2024-04-02T16:22:28.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.124+0000 7f1543fff640 1 -- 172.21.15.50:0/3513188076 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f154c059c50 con 0x7f155c156700 2024-04-02T16:22:28.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.125+0000 7f722affd640 1 -- 172.21.15.50:0/3514799601 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f722c046020 con 0x7f7234161b40 2024-04-02T16:22:28.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.126+0000 7f722affd640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f721806c940 0x7f721806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.126+0000 7fa1a17fa640 1 -- 172.21.15.50:0/2560461846 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa18c056ac0 con 0x7fa1a41f32c0 2024-04-02T16:22:28.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.126+0000 7f147effd640 1 -- 172.21.15.50:0/4123107748 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14880c2060 con 0x7f14800df1e0 2024-04-02T16:22:28.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.126+0000 7f723259c640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f721806c940 0x7f721806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.126+0000 7f147effd640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f146c06c940 0x7f146c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.127+0000 7f723259c640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f721806c940 0x7f721806ee00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f721c005880 tx=0x7f721c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.128+0000 7f148e318640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f146c06c940 0x7f146c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.128+0000 7f148e318640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f146c06c940 0x7f146c06ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1484005880 tx=0x7f1484009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.130+0000 7f722affd640 1 -- 172.21.15.50:0/3514799601 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f722c07a4d0 con 0x7f7234161b40 2024-04-02T16:22:28.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.131+0000 7f147effd640 1 -- 172.21.15.50:0/4123107748 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f14880f4370 con 0x7f14800df1e0 2024-04-02T16:22:28.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.131+0000 7f5ebffff640 1 -- 172.21.15.50:0/3744092765 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5ecc058bf0 con 0x7f5ed806e670 2024-04-02T16:22:28.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.136+0000 7f722affd640 1 -- 172.21.15.50:0/3514799601 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f722c04d070 con 0x7f7234161b40 2024-04-02T16:22:28.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.145+0000 7f147effd640 1 -- 172.21.15.50:0/4123107748 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f14880f69d0 con 0x7f14800df1e0 2024-04-02T16:22:28.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.484+0000 7f8e79b82640 1 -- 172.21.15.50:0/1985302205 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8e6c005390 con 0x7f8e6c054260 2024-04-02T16:22:28.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.485+0000 7f8e717fa640 1 -- 172.21.15.50:0/1985302205 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e6805e160 con 0x7f8e6c054260 2024-04-02T16:22:28.486 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:28.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.488+0000 7f8e5effd640 1 -- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8e5806c9c0 msgr2=0x7f8e5806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.488+0000 7f8e5effd640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8e5806c9c0 0x7f8e5806ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c100720 tx=0x7f8e60009040 comp rx=0 tx=0).stop 2024-04-02T16:22:28.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.488+0000 7f8e5effd640 1 -- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e6c054260 msgr2=0x7f8e6c0f8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.489+0000 7f8e5effd640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e6c054260 0x7f8e6c0f8df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800fa70 tx=0x7f8e6800ff30 comp rx=0 tx=0).stop 2024-04-02T16:22:28.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.489+0000 7f8e5effd640 1 -- 172.21.15.50:0/1985302205 shutdown_connections 2024-04-02T16:22:28.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.489+0000 7f8e5effd640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8e5806c9c0 0x7f8e5806ee80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.489+0000 7f8e5effd640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e6c101010 0x7f8e6c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.489+0000 7f8e5effd640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e6c054260 0x7f8e6c0f8df0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.489+0000 7f8e5effd640 1 --2- 172.21.15.50:0/1985302205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e6c033920 0x7f8e6c04d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.489+0000 7f8e5effd640 1 -- 172.21.15.50:0/1985302205 >> 172.21.15.50:0/1985302205 conn(0x7f8e6c009fb0 msgr2=0x7f8e6c034520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.490+0000 7f8e5effd640 1 -- 172.21.15.50:0/1985302205 shutdown_connections 2024-04-02T16:22:28.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.490+0000 7f8e5effd640 1 -- 172.21.15.50:0/1985302205 wait complete. 2024-04-02T16:22:28.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.490+0000 7fbfc359e640 1 -- 172.21.15.50:0/135690245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbfc40756f0 con 0x7fbfc415a220 2024-04-02T16:22:28.498 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.495+0000 7fbfb2ffd640 1 -- 172.21.15.50:0/135690245 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfbc022020 con 0x7fbfc415a220 2024-04-02T16:22:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.498+0000 7fbfb0ff9640 1 -- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbfac06cb00 msgr2=0x7fbfac06efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.498+0000 7fbfb0ff9640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbfac06cb00 0x7fbfac06efc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb80090e0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.498+0000 7fbfb0ff9640 1 -- 172.21.15.50:0/135690245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 msgr2=0x7fbfc406db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.498+0000 7fbfb0ff9640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc406db50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00c8c0 tx=0x7fbfbc00cd80 comp rx=0 tx=0).stop 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 -- 172.21.15.50:0/135690245 shutdown_connections 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbfac06cb00 0x7fbfac06efc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbfc415a220 0x7fbfc406db50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfc40d0770 0x7fbfc414b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 --2- 172.21.15.50:0/135690245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfc40ce230 0x7fbfc4155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 -- 172.21.15.50:0/135690245 >> 172.21.15.50:0/135690245 conn(0x7fbfc40cfc50 msgr2=0x7fbfc4156540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 -- 172.21.15.50:0/135690245 shutdown_connections 2024-04-02T16:22:28.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.499+0000 7fbfb0ff9640 1 -- 172.21.15.50:0/135690245 wait complete. 2024-04-02T16:22:28.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-02T16:22:28.511 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-04-02T16:22:28.517 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.527+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa1a4073810 con 0x7fa1a41f32c0 2024-04-02T16:22:28.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.528+0000 7fa1a17fa640 1 -- 172.21.15.50:0/2560461846 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1a406e1c0 con 0x7fa1a41f32c0 2024-04-02T16:22:28.530 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa18406c970 msgr2=0x7fa18406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa18406c970 0x7fa18406ee30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa19c007fd0 tx=0x7fa19c02f000 comp rx=0 tx=0).stop 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a41f32c0 msgr2=0x7fa1a40c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a41f32c0 0x7fa1a40c53e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa18c011cd0 tx=0x7fa18c00b510 comp rx=0 tx=0).stop 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 shutdown_connections 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a406d8c0 0x7fa1a40c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a41f32c0 0x7fa1a40c53e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa18406c970 0x7fa18406ee30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 --2- 172.21.15.50:0/2560461846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa190004c40 0x7fa1a40c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 >> 172.21.15.50:0/2560461846 conn(0x7fa1a4072b40 msgr2=0x7fa1a40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 shutdown_connections 2024-04-02T16:22:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.530+0000 7fa1ab0ba640 1 -- 172.21.15.50:0/2560461846 wait complete. 2024-04-02T16:22:28.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.538+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5eb8005580 con 0x7f5ed806e670 2024-04-02T16:22:28.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.538+0000 7f5ebffff640 1 -- 172.21.15.50:0/3744092765 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ecc05cef0 con 0x7f5ed806e670 2024-04-02T16:22:28.539 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5ec406c8f0 msgr2=0x7f5ec406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5ec406c8f0 0x7f5ec406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0035000 tx=0x7f5ed0030000 comp rx=0 tx=0).stop 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ed806e670 msgr2=0x7f5ed8165980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ed806e670 0x7f5ed8165980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c400 tx=0x7f5ecc00c8c0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 shutdown_connections 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed80c9e70 0x7f5ed80ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed8162a50 0x7f5ed80c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5ec406c8f0 0x7f5ec406edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 --2- 172.21.15.50:0/3744092765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ed806e670 0x7f5ed8165980 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 >> 172.21.15.50:0/3744092765 conn(0x7f5ed80d0170 msgr2=0x7f5ed8072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 shutdown_connections 2024-04-02T16:22:28.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.541+0000 7f5edcf6f640 1 -- 172.21.15.50:0/3744092765 wait complete. 2024-04-02T16:22:28.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-04-02T16:22:28.546 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.547+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe8840cad20 con 0x7fe88415e5d0 2024-04-02T16:22:28.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.548+0000 7fe87a7fc640 1 -- 172.21.15.50:0/2382718043 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe88005ce00 con 0x7fe88415e5d0 2024-04-02T16:22:28.549 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe86806c9f0 msgr2=0x7fe86806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe86806c9f0 0x7fe86806eeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe874006fd0 tx=0x7fe874008040 comp rx=0 tx=0).stop 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88415e5d0 msgr2=0x7fe8840b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88415e5d0 0x7fe8840b3c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c390 tx=0x7fe88000c790 comp rx=0 tx=0).stop 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 shutdown_connections 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe86806c9f0 0x7fe86806eeb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe88415e5d0 0x7fe8840b3c50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8840d46b0 0x7fe88415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 --2- 172.21.15.50:0/2382718043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8840cf210 0x7fe8840b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.551+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 >> 172.21.15.50:0/2382718043 conn(0x7fe88408dce0 msgr2=0x7fe88415ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.552+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 shutdown_connections 2024-04-02T16:22:28.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.552+0000 7fe88bd15640 1 -- 172.21.15.50:0/2382718043 wait complete. 2024-04-02T16:22:28.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-02T16:22:28.555 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.555+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1534005850 con 0x7f155c156700 2024-04-02T16:22:28.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.556+0000 7f1543fff640 1 -- 172.21.15.50:0/3513188076 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f154c059550 con 0x7f155c156700 2024-04-02T16:22:28.557 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:28.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f154806c940 msgr2=0x7f154806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f154806c940 0x7f154806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1550009fb0 tx=0x7f1550009190 comp rx=0 tx=0).stop 2024-04-02T16:22:28.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c156700 msgr2=0x7f155c0c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c156700 0x7f155c0c3790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f155c163d50 tx=0x7f154c00f8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 shutdown_connections 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c0b5b00 0x7f155c0b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c156700 0x7f155c0c3790 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f154806c940 0x7f154806ee00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 --2- 172.21.15.50:0/3513188076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15440051c0 0x7f155c0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 >> 172.21.15.50:0/3513188076 conn(0x7f155c0d0de0 msgr2=0x7f155c0b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 shutdown_connections 2024-04-02T16:22:28.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.558+0000 7f1541ffb640 1 -- 172.21.15.50:0/3513188076 wait complete. 2024-04-02T16:22:28.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-02T16:22:28.564 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-02T16:22:28.570 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.572+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f72340d1770 con 0x7f7234161b40 2024-04-02T16:22:28.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.572+0000 7f722affd640 1 -- 172.21.15.50:0/3514799601 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f722c032340 con 0x7f7234161b40 2024-04-02T16:22:28.573 INFO:teuthology.orchestra.run.smithi050.stdout:55834574857 2024-04-02T16:22:28.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f721806c940 msgr2=0x7f721806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f721806c940 0x7f721806ee00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f721c005880 tx=0x7f721c009040 comp rx=0 tx=0).stop 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7234161b40 msgr2=0x7f723415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7234161b40 0x7f723415abd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f722c033000 tx=0x7f722c004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 shutdown_connections 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f721806c940 0x7f721806ee00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7234161b40 0x7f723415abd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7234161760 0x7f72340c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 --2- 172.21.15.50:0/3514799601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72340d1160 0x7f72341514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 >> 172.21.15.50:0/3514799601 conn(0x7f723409d460 msgr2=0x7f72340b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 shutdown_connections 2024-04-02T16:22:28.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.575+0000 7f7238b11640 1 -- 172.21.15.50:0/3514799601 wait complete. 2024-04-02T16:22:28.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.579+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1480036250 con 0x7f14800df1e0 2024-04-02T16:22:28.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.580+0000 7f147effd640 1 -- 172.21.15.50:0/4123107748 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14880f62d0 con 0x7f14800df1e0 2024-04-02T16:22:28.581 INFO:teuthology.orchestra.run.smithi050.stdout:55834574858 2024-04-02T16:22:28.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f146c06c940 msgr2=0x7f146c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f146c06c940 0x7f146c06ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1484005880 tx=0x7f1484009040 comp rx=0 tx=0).stop 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14800df1e0 msgr2=0x7f14800ddea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14800df1e0 0x7f14800ddea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f148806a330 tx=0x7f148806b3e0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 shutdown_connections 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f146c06c940 0x7f146c06ee00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14800df1e0 0x7f14800ddea0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1480053c30 0x7f14800dd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 --2- 172.21.15.50:0/4123107748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1480050160 0x7f14800db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 >> 172.21.15.50:0/4123107748 conn(0x7f1480009f70 msgr2=0x7f148004fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 shutdown_connections 2024-04-02T16:22:28.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.583+0000 7f148f31a640 1 -- 172.21.15.50:0/4123107748 wait complete. 2024-04-02T16:22:28.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-02T16:22:28.591 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-02T16:22:28.596 DEBUG:teuthology.parallel:result is None 2024-04-02T16:22:28.596 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-02T16:22:28.596 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:22:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.737+0000 7f57280dc640 1 Processor -- start 2024-04-02T16:22:28.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.737+0000 7f57280dc640 1 -- start start 2024-04-02T16:22:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.738+0000 7f57280dc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f572015e570 0x7f57200cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.738+0000 7f57280dc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 0x7f57200c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.738+0000 7f57280dc640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57200d05b0 con 0x7f5720162f70 2024-04-02T16:22:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.738+0000 7f57280dc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f57200d11c0 con 0x7f572015e570 2024-04-02T16:22:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.738+0000 7f57280dc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f57200d0890 con 0x7f57200d0130 2024-04-02T16:22:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.738+0000 7f5725677640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5720162f70 0x7f5720162340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55098/0 (socket says 172.21.15.50:55098) 2024-04-02T16:22:28.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.738+0000 7f5725677640 1 -- 172.21.15.50:0/2442883704 learned_addr learned my addr 172.21.15.50:0/2442883704 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5726679640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 0x7f57200c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725e78640 1 --2- 172.21.15.50:0/2442883704 >> v2:172.21.15.188:3300/0 conn(0x7f572015e570 0x7f57200cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563710731 0 0) 0x7f57200d05b0 con 0x7f5720162f70 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5708003520 con 0x7f5720162f70 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1321026648 0 0) 0x7f5708003520 con 0x7f5720162f70 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 msgr2=0x7f57200c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 0x7f57200c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 >> v2:172.21.15.188:3300/0 conn(0x7f572015e570 msgr2=0x7f57200cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 --2- 172.21.15.50:0/2442883704 >> v2:172.21.15.188:3300/0 conn(0x7f572015e570 0x7f57200cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57200d0b70 con 0x7f5720162f70 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f5714001690 con 0x7f5720162f70 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 >> v1:172.21.15.50:6789/0 conn(0x7f5720162f70 legacy=0x7f5720162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080039b0 0x7f5708003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5708004800 0x7f5708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 0x7f57080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57200d11c0 con 0x7f57080039b0 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57200d0890 con 0x7f5708004800 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f570800b9c0 con 0x7f57200d0130 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725e78640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080039b0 0x7f5708003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5726679640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 0x7f57080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725677640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5708004800 0x7f5708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725e78640 1 -- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 msgr2=0x7f57080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725e78640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57200d0130 0x7f57080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725e78640 1 -- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5708004800 msgr2=0x7f5708004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725e78640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5708004800 0x7f5708004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.739+0000 7f5725e78640 1 -- 172.21.15.50:0/2442883704 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5708003520 con 0x7f57080039b0 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.740+0000 7f5725e78640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080039b0 0x7f5708003dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5710009fc0 tx=0x7f5708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7170c48cff2eb45d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.740+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710039070 con 0x7f57080039b0 2024-04-02T16:22:28.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.740+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571002cc60 con 0x7f57080039b0 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.740+0000 7f5724e76640 1 -- 172.21.15.50:0/2442883704 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710009c10 con 0x7f57080039b0 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.741+0000 7f57280dc640 1 -- 172.21.15.50:0/2442883704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080039b0 msgr2=0x7f5708003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.741+0000 7f57280dc640 1 --2- 172.21.15.50:0/2442883704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57080039b0 0x7f5708003dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5710009fc0 tx=0x7f5708005830 comp rx=0 tx=0).stop 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.741+0000 7f5726679640 1 -- 172.21.15.50:0/2442883704 reap_dead start 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.741+0000 7f57280dc640 1 -- 172.21.15.50:0/2442883704 shutdown_connections 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.741+0000 7f57280dc640 1 -- 172.21.15.50:0/2442883704 >> 172.21.15.50:0/2442883704 conn(0x7f572009d5a0 msgr2=0x7f57200d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.741+0000 7f57280dc640 1 -- 172.21.15.50:0/2442883704 shutdown_connections 2024-04-02T16:22:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.741+0000 7f57280dc640 1 -- 172.21.15.50:0/2442883704 wait complete. 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.742+0000 7f57280dc640 1 Processor -- start 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.742+0000 7f57280dc640 1 -- start start 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f57280dc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57080039b0 0x7f5720151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f57280dc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57200b9b60 0x7f57200b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f57280dc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5720160a30 0x7f5720160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f57280dc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5720162bd0 con 0x7f57200b9b60 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f57280dc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5720162a50 con 0x7f57080039b0 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f57280dc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f57200d1650 con 0x7f5720160a30 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57200b9b60 0x7f57200b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57200b9b60 0x7f57200b7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45350/0 (socket says 172.21.15.50:45350) 2024-04-02T16:22:28.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 -- 172.21.15.50:0/479951627 learned_addr learned my addr 172.21.15.50:0/479951627 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725e78640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57080039b0 0x7f5720151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5726679640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5720160a30 0x7f5720160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 -- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5720160a30 msgr2=0x7f5720160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5720160a30 0x7f5720160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 -- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57080039b0 msgr2=0x7f5720151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57080039b0 0x7f5720151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 -- 172.21.15.50:0/479951627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572014ef50 con 0x7f57200b9b60 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725e78640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57080039b0 0x7f5720151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5726679640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5720160a30 0x7f5720160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.743+0000 7f5725677640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57200b9b60 0x7f57200b7810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5714007370 tx=0x7f5714007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.744+0000 7f571e7fc640 1 -- 172.21.15.50:0/479951627 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571400f960 con 0x7f57200b9b60 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.744+0000 7f571e7fc640 1 -- 172.21.15.50:0/479951627 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5714018070 con 0x7f57200b9b60 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.744+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5720151f50 con 0x7f57200b9b60 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.744+0000 7f571e7fc640 1 -- 172.21.15.50:0/479951627 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571400fd20 con 0x7f57200b9b60 2024-04-02T16:22:28.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.744+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57200b8490 con 0x7f57200b9b60 2024-04-02T16:22:28.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.745+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57200756f0 con 0x7f57200b9b60 2024-04-02T16:22:28.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.745+0000 7f571e7fc640 1 -- 172.21.15.50:0/479951627 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f571401c410 con 0x7f57200b9b60 2024-04-02T16:22:28.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.745+0000 7f571e7fc640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f570c06c920 0x7f570c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:28.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.746+0000 7f571e7fc640 1 -- 172.21.15.50:0/479951627 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f571408baf0 con 0x7f57200b9b60 2024-04-02T16:22:28.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.746+0000 7f5725e78640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f570c06c920 0x7f570c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:28.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.746+0000 7f5725e78640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f570c06c920 0x7f570c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5710036000 tx=0x7f571002e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:28.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.750+0000 7f571e7fc640 1 -- 172.21.15.50:0/479951627 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5714059c70 con 0x7f57200b9b60 2024-04-02T16:22:28.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.931+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 --> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5720160810 con 0x7f570c06c920 2024-04-02T16:22:28.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.940+0000 7f571e7fc640 1 -- 172.21.15.50:0/479951627 <== mgr.4112 v2:172.21.15.188:6832/2785600878 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f5720160810 con 0x7f570c06c920 2024-04-02T16:22:28.946 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:28.947 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-04-02T16:22:28.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f570c06c920 msgr2=0x7f570c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f570c06c920 0x7f570c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5710036000 tx=0x7f571002e000 comp rx=0 tx=0).stop 2024-04-02T16:22:28.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57200b9b60 msgr2=0x7f57200b7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:28.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57200b9b60 0x7f57200b7810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5714007370 tx=0x7f5714007830 comp rx=0 tx=0).stop 2024-04-02T16:22:28.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 shutdown_connections 2024-04-02T16:22:28.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5720160a30 0x7f5720160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f570c06c920 0x7f570c06ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57200b9b60 0x7f57200b7810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 --2- 172.21.15.50:0/479951627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57080039b0 0x7f5720151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:28.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 >> 172.21.15.50:0/479951627 conn(0x7f572009d5a0 msgr2=0x7f57200c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:28.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 shutdown_connections 2024-04-02T16:22:28.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:28.943+0000 7f57280dc640 1 -- 172.21.15.50:0/479951627 wait complete. 2024-04-02T16:22:28.955 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-02T16:22:28.672801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236104,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738616,"statfs":{"total":773094113280,"available":772852342784,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003105"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434993+0000","last_change":"2024-04-02T16:22:00.392396+0000","last_active":"2024-04-02T16:22:07.434993+0000","last_peered":"2024-04-02T16:22:07.434993+0000","last_clean":"2024-04-02T16:22:07.434993+0000","last_became_active":"2024-04-02T16:22:00.392089+0000","last_became_peered":"2024-04-02T16:22:00.392089+0000","last_unstale":"2024-04-02T16:22:07.434993+0000","last_undegraded":"2024-04-02T16:22:07.434993+0000","last_fullsized":"2024-04-02T16:22:07.434993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428539+0000","last_change":"2024-04-02T16:22:01.410877+0000","last_active":"2024-04-02T16:22:07.428539+0000","last_peered":"2024-04-02T16:22:07.428539+0000","last_clean":"2024-04-02T16:22:07.428539+0000","last_became_active":"2024-04-02T16:22:01.410574+0000","last_became_peered":"2024-04-02T16:22:01.410574+0000","last_unstale":"2024-04-02T16:22:07.428539+0000","last_undegraded":"2024-04-02T16:22:07.428539+0000","last_fullsized":"2024-04-02T16:22:07.428539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427767+0000","last_change":"2024-04-02T16:22:05.817506+0000","last_active":"2024-04-02T16:22:07.427767+0000","last_peered":"2024-04-02T16:22:07.427767+0000","last_clean":"2024-04-02T16:22:07.427767+0000","last_became_active":"2024-04-02T16:22:05.817145+0000","last_became_peered":"2024-04-02T16:22:05.817145+0000","last_unstale":"2024-04-02T16:22:07.427767+0000","last_undegraded":"2024-04-02T16:22:07.427767+0000","last_fullsized":"2024-04-02T16:22:07.427767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:30.473603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428664+0000","last_change":"2024-04-02T16:22:00.391720+0000","last_active":"2024-04-02T16:22:07.428664+0000","last_peered":"2024-04-02T16:22:07.428664+0000","last_clean":"2024-04-02T16:22:07.428664+0000","last_became_active":"2024-04-02T16:22:00.391262+0000","last_became_peered":"2024-04-02T16:22:00.391262+0000","last_unstale":"2024-04-02T16:22:07.428664+0000","last_undegraded":"2024-04-02T16:22:07.428664+0000","last_fullsized":"2024-04-02T16:22:07.428664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434717+0000","last_change":"2024-04-02T16:22:01.403096+0000","last_active":"2024-04-02T16:22:07.434717+0000","last_peered":"2024-04-02T16:22:07.434717+0000","last_clean":"2024-04-02T16:22:07.434717+0000","last_became_active":"2024-04-02T16:22:01.402805+0000","last_became_peered":"2024-04-02T16:22:01.402805+0000","last_unstale":"2024-04-02T16:22:07.434717+0000","last_undegraded":"2024-04-02T16:22:07.434717+0000","last_fullsized":"2024-04-02T16:22:07.434717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428967+0000","last_change":"2024-04-02T16:22:05.447225+0000","last_active":"2024-04-02T16:22:07.428967+0000","last_peered":"2024-04-02T16:22:07.428967+0000","last_clean":"2024-04-02T16:22:07.428967+0000","last_became_active":"2024-04-02T16:22:05.446843+0000","last_became_peered":"2024-04-02T16:22:05.446843+0000","last_unstale":"2024-04-02T16:22:07.428967+0000","last_undegraded":"2024-04-02T16:22:07.428967+0000","last_fullsized":"2024-04-02T16:22:07.428967+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:00.390875+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:00.390501+0000","last_became_peered":"2024-04-02T16:22:00.390501+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430302+0000","last_change":"2024-04-02T16:22:01.410933+0000","last_active":"2024-04-02T16:22:07.430302+0000","last_peered":"2024-04-02T16:22:07.430302+0000","last_clean":"2024-04-02T16:22:07.430302+0000","last_became_active":"2024-04-02T16:22:01.410242+0000","last_became_peered":"2024-04-02T16:22:01.410242+0000","last_unstale":"2024-04-02T16:22:07.430302+0000","last_undegraded":"2024-04-02T16:22:07.430302+0000","last_fullsized":"2024-04-02T16:22:07.430302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429559+0000","last_change":"2024-04-02T16:22:05.440230+0000","last_active":"2024-04-02T16:22:07.429559+0000","last_peered":"2024-04-02T16:22:07.429559+0000","last_clean":"2024-04-02T16:22:07.429559+0000","last_became_active":"2024-04-02T16:22:05.439599+0000","last_became_peered":"2024-04-02T16:22:05.439599+0000","last_unstale":"2024-04-02T16:22:07.429559+0000","last_undegraded":"2024-04-02T16:22:07.429559+0000","last_fullsized":"2024-04-02T16:22:07.429559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:35.350953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434504+0000","last_change":"2024-04-02T16:22:00.392521+0000","last_active":"2024-04-02T16:22:07.434504+0000","last_peered":"2024-04-02T16:22:07.434504+0000","last_clean":"2024-04-02T16:22:07.434504+0000","last_became_active":"2024-04-02T16:22:00.392244+0000","last_became_peered":"2024-04-02T16:22:00.392244+0000","last_unstale":"2024-04-02T16:22:07.434504+0000","last_undegraded":"2024-04-02T16:22:07.434504+0000","last_fullsized":"2024-04-02T16:22:07.434504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430629+0000","last_change":"2024-04-02T16:22:01.406612+0000","last_active":"2024-04-02T16:22:07.430629+0000","last_peered":"2024-04-02T16:22:07.430629+0000","last_clean":"2024-04-02T16:22:07.430629+0000","last_became_active":"2024-04-02T16:22:01.404789+0000","last_became_peered":"2024-04-02T16:22:01.404789+0000","last_unstale":"2024-04-02T16:22:07.430629+0000","last_undegraded":"2024-04-02T16:22:07.430629+0000","last_fullsized":"2024-04-02T16:22:07.430629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429046+0000","last_change":"2024-04-02T16:22:05.445479+0000","last_active":"2024-04-02T16:22:07.429046+0000","last_peered":"2024-04-02T16:22:07.429046+0000","last_clean":"2024-04-02T16:22:07.429046+0000","last_became_active":"2024-04-02T16:22:05.445022+0000","last_became_peered":"2024-04-02T16:22:05.445022+0000","last_unstale":"2024-04-02T16:22:07.429046+0000","last_undegraded":"2024-04-02T16:22:07.429046+0000","last_fullsized":"2024-04-02T16:22:07.429046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.404940+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.404670+0000","last_became_peered":"2024-04-02T16:22:01.404670+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434832+0000","last_change":"2024-04-02T16:22:00.390811+0000","last_active":"2024-04-02T16:22:07.434832+0000","last_peered":"2024-04-02T16:22:07.434832+0000","last_clean":"2024-04-02T16:22:07.434832+0000","last_became_active":"2024-04-02T16:22:00.390515+0000","last_became_peered":"2024-04-02T16:22:00.390515+0000","last_unstale":"2024-04-02T16:22:07.434832+0000","last_undegraded":"2024-04-02T16:22:07.434832+0000","last_fullsized":"2024-04-02T16:22:07.434832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429947+0000","last_change":"2024-04-02T16:22:05.445431+0000","last_active":"2024-04-02T16:22:07.429947+0000","last_peered":"2024-04-02T16:22:07.429947+0000","last_clean":"2024-04-02T16:22:07.429947+0000","last_became_active":"2024-04-02T16:22:05.445014+0000","last_became_peered":"2024-04-02T16:22:05.445014+0000","last_unstale":"2024-04-02T16:22:07.429947+0000","last_undegraded":"2024-04-02T16:22:07.429947+0000","last_fullsized":"2024-04-02T16:22:07.429947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428171+0000","last_change":"2024-04-02T16:22:01.409289+0000","last_active":"2024-04-02T16:22:07.428171+0000","last_peered":"2024-04-02T16:22:07.428171+0000","last_clean":"2024-04-02T16:22:07.428171+0000","last_became_active":"2024-04-02T16:22:01.408934+0000","last_became_peered":"2024-04-02T16:22:01.408934+0000","last_unstale":"2024-04-02T16:22:07.428171+0000","last_undegraded":"2024-04-02T16:22:07.428171+0000","last_fullsized":"2024-04-02T16:22:07.428171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434963+0000","last_change":"2024-04-02T16:22:00.390727+0000","last_active":"2024-04-02T16:22:07.434963+0000","last_peered":"2024-04-02T16:22:07.434963+0000","last_clean":"2024-04-02T16:22:07.434963+0000","last_became_active":"2024-04-02T16:22:00.390131+0000","last_became_peered":"2024-04-02T16:22:00.390131+0000","last_unstale":"2024-04-02T16:22:07.434963+0000","last_undegraded":"2024-04-02T16:22:07.434963+0000","last_fullsized":"2024-04-02T16:22:07.434963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429330+0000","last_change":"2024-04-02T16:22:05.447195+0000","last_active":"2024-04-02T16:22:07.429330+0000","last_peered":"2024-04-02T16:22:07.429330+0000","last_clean":"2024-04-02T16:22:07.429330+0000","last_became_active":"2024-04-02T16:22:05.446753+0000","last_became_peered":"2024-04-02T16:22:05.446753+0000","last_unstale":"2024-04-02T16:22:07.429330+0000","last_undegraded":"2024-04-02T16:22:07.429330+0000","last_fullsized":"2024-04-02T16:22:07.429330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427951+0000","last_change":"2024-04-02T16:22:01.400634+0000","last_active":"2024-04-02T16:22:07.427951+0000","last_peered":"2024-04-02T16:22:07.427951+0000","last_clean":"2024-04-02T16:22:07.427951+0000","last_became_active":"2024-04-02T16:22:01.400309+0000","last_became_peered":"2024-04-02T16:22:01.400309+0000","last_unstale":"2024-04-02T16:22:07.427951+0000","last_undegraded":"2024-04-02T16:22:07.427951+0000","last_fullsized":"2024-04-02T16:22:07.427951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434579+0000","last_change":"2024-04-02T16:22:00.391494+0000","last_active":"2024-04-02T16:22:07.434579+0000","last_peered":"2024-04-02T16:22:07.434579+0000","last_clean":"2024-04-02T16:22:07.434579+0000","last_became_active":"2024-04-02T16:22:00.391190+0000","last_became_peered":"2024-04-02T16:22:00.391190+0000","last_unstale":"2024-04-02T16:22:07.434579+0000","last_undegraded":"2024-04-02T16:22:07.434579+0000","last_fullsized":"2024-04-02T16:22:07.434579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435264+0000","last_change":"2024-04-02T16:22:05.811534+0000","last_active":"2024-04-02T16:22:07.435264+0000","last_peered":"2024-04-02T16:22:07.435264+0000","last_clean":"2024-04-02T16:22:07.435264+0000","last_became_active":"2024-04-02T16:22:05.811134+0000","last_became_peered":"2024-04-02T16:22:05.811134+0000","last_unstale":"2024-04-02T16:22:07.435264+0000","last_undegraded":"2024-04-02T16:22:07.435264+0000","last_fullsized":"2024-04-02T16:22:07.435264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430330+0000","last_change":"2024-04-02T16:22:01.399287+0000","last_active":"2024-04-02T16:22:07.430330+0000","last_peered":"2024-04-02T16:22:07.430330+0000","last_clean":"2024-04-02T16:22:07.430330+0000","last_became_active":"2024-04-02T16:22:01.398956+0000","last_became_peered":"2024-04-02T16:22:01.398956+0000","last_unstale":"2024-04-02T16:22:07.430330+0000","last_undegraded":"2024-04-02T16:22:07.430330+0000","last_fullsized":"2024-04-02T16:22:07.430330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:50.927884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434912+0000","last_change":"2024-04-02T16:22:00.391289+0000","last_active":"2024-04-02T16:22:07.434912+0000","last_peered":"2024-04-02T16:22:07.434912+0000","last_clean":"2024-04-02T16:22:07.434912+0000","last_became_active":"2024-04-02T16:22:00.390292+0000","last_became_peered":"2024-04-02T16:22:00.390292+0000","last_unstale":"2024-04-02T16:22:07.434912+0000","last_undegraded":"2024-04-02T16:22:07.434912+0000","last_fullsized":"2024-04-02T16:22:07.434912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429938+0000","last_change":"2024-04-02T16:22:05.440165+0000","last_active":"2024-04-02T16:22:07.429938+0000","last_peered":"2024-04-02T16:22:07.429938+0000","last_clean":"2024-04-02T16:22:07.429938+0000","last_became_active":"2024-04-02T16:22:05.439732+0000","last_became_peered":"2024-04-02T16:22:05.439732+0000","last_unstale":"2024-04-02T16:22:07.429938+0000","last_undegraded":"2024-04-02T16:22:07.429938+0000","last_fullsized":"2024-04-02T16:22:07.429938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434684+0000","last_change":"2024-04-02T16:22:01.404915+0000","last_active":"2024-04-02T16:22:07.434684+0000","last_peered":"2024-04-02T16:22:07.434684+0000","last_clean":"2024-04-02T16:22:07.434684+0000","last_became_active":"2024-04-02T16:22:01.402769+0000","last_became_peered":"2024-04-02T16:22:01.402769+0000","last_unstale":"2024-04-02T16:22:07.434684+0000","last_undegraded":"2024-04-02T16:22:07.434684+0000","last_fullsized":"2024-04-02T16:22:07.434684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:45.810039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435410+0000","last_change":"2024-04-02T16:22:00.393479+0000","last_active":"2024-04-02T16:22:07.435410+0000","last_peered":"2024-04-02T16:22:07.435410+0000","last_clean":"2024-04-02T16:22:07.435410+0000","last_became_active":"2024-04-02T16:22:00.393178+0000","last_became_peered":"2024-04-02T16:22:00.393178+0000","last_unstale":"2024-04-02T16:22:07.435410+0000","last_undegraded":"2024-04-02T16:22:07.435410+0000","last_fullsized":"2024-04-02T16:22:07.435410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430212+0000","last_change":"2024-04-02T16:22:05.815460+0000","last_active":"2024-04-02T16:22:07.430212+0000","last_peered":"2024-04-02T16:22:07.430212+0000","last_clean":"2024-04-02T16:22:07.430212+0000","last_became_active":"2024-04-02T16:22:05.814988+0000","last_became_peered":"2024-04-02T16:22:05.814988+0000","last_unstale":"2024-04-02T16:22:07.430212+0000","last_undegraded":"2024-04-02T16:22:07.430212+0000","last_fullsized":"2024-04-02T16:22:07.430212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:01.404044+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:01.403758+0000","last_became_peered":"2024-04-02T16:22:01.403758+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435274+0000","last_change":"2024-04-02T16:22:00.394933+0000","last_active":"2024-04-02T16:22:07.435274+0000","last_peered":"2024-04-02T16:22:07.435274+0000","last_clean":"2024-04-02T16:22:07.435274+0000","last_became_active":"2024-04-02T16:22:00.394672+0000","last_became_peered":"2024-04-02T16:22:00.394672+0000","last_unstale":"2024-04-02T16:22:07.435274+0000","last_undegraded":"2024-04-02T16:22:07.435274+0000","last_fullsized":"2024-04-02T16:22:07.435274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427719+0000","last_change":"2024-04-02T16:22:05.816404+0000","last_active":"2024-04-02T16:22:07.427719+0000","last_peered":"2024-04-02T16:22:07.427719+0000","last_clean":"2024-04-02T16:22:07.427719+0000","last_became_active":"2024-04-02T16:22:05.816011+0000","last_became_peered":"2024-04-02T16:22:05.816011+0000","last_unstale":"2024-04-02T16:22:07.427719+0000","last_undegraded":"2024-04-02T16:22:07.427719+0000","last_fullsized":"2024-04-02T16:22:07.427719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:58.796179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:01.405660+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:01.405343+0000","last_became_peered":"2024-04-02T16:22:01.405343+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435169+0000","last_change":"2024-04-02T16:22:00.391246+0000","last_active":"2024-04-02T16:22:07.435169+0000","last_peered":"2024-04-02T16:22:07.435169+0000","last_clean":"2024-04-02T16:22:07.435169+0000","last_became_active":"2024-04-02T16:22:00.390836+0000","last_became_peered":"2024-04-02T16:22:00.390836+0000","last_unstale":"2024-04-02T16:22:07.435169+0000","last_undegraded":"2024-04-02T16:22:07.435169+0000","last_fullsized":"2024-04-02T16:22:07.435169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:22.952130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435359+0000","last_change":"2024-04-02T16:22:05.815150+0000","last_active":"2024-04-02T16:22:07.435359+0000","last_peered":"2024-04-02T16:22:07.435359+0000","last_clean":"2024-04-02T16:22:07.435359+0000","last_became_active":"2024-04-02T16:22:05.814788+0000","last_became_peered":"2024-04-02T16:22:05.814788+0000","last_unstale":"2024-04-02T16:22:07.435359+0000","last_undegraded":"2024-04-02T16:22:07.435359+0000","last_fullsized":"2024-04-02T16:22:07.435359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430743+0000","last_change":"2024-04-02T16:22:01.404482+0000","last_active":"2024-04-02T16:22:07.430743+0000","last_peered":"2024-04-02T16:22:07.430743+0000","last_clean":"2024-04-02T16:22:07.430743+0000","last_became_active":"2024-04-02T16:22:01.402612+0000","last_became_peered":"2024-04-02T16:22:01.402612+0000","last_unstale":"2024-04-02T16:22:07.430743+0000","last_undegraded":"2024-04-02T16:22:07.430743+0000","last_fullsized":"2024-04-02T16:22:07.430743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435056+0000","last_change":"2024-04-02T16:22:00.388850+0000","last_active":"2024-04-02T16:22:07.435056+0000","last_peered":"2024-04-02T16:22:07.435056+0000","last_clean":"2024-04-02T16:22:07.435056+0000","last_became_active":"2024-04-02T16:22:00.388569+0000","last_became_peered":"2024-04-02T16:22:00.388569+0000","last_unstale":"2024-04-02T16:22:07.435056+0000","last_undegraded":"2024-04-02T16:22:07.435056+0000","last_fullsized":"2024-04-02T16:22:07.435056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434575+0000","last_change":"2024-04-02T16:22:05.445952+0000","last_active":"2024-04-02T16:22:07.434575+0000","last_peered":"2024-04-02T16:22:07.434575+0000","last_clean":"2024-04-02T16:22:07.434575+0000","last_became_active":"2024-04-02T16:22:05.445442+0000","last_became_peered":"2024-04-02T16:22:05.445442+0000","last_unstale":"2024-04-02T16:22:07.434575+0000","last_undegraded":"2024-04-02T16:22:07.434575+0000","last_fullsized":"2024-04-02T16:22:07.434575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429867+0000","last_change":"2024-04-02T16:22:01.405033+0000","last_active":"2024-04-02T16:22:07.429867+0000","last_peered":"2024-04-02T16:22:07.429867+0000","last_clean":"2024-04-02T16:22:07.429867+0000","last_became_active":"2024-04-02T16:22:01.404647+0000","last_became_peered":"2024-04-02T16:22:01.404647+0000","last_unstale":"2024-04-02T16:22:07.429867+0000","last_undegraded":"2024-04-02T16:22:07.429867+0000","last_fullsized":"2024-04-02T16:22:07.429867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:03.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429937+0000","last_change":"2024-04-02T16:22:00.390557+0000","last_active":"2024-04-02T16:22:07.429937+0000","last_peered":"2024-04-02T16:22:07.429937+0000","last_clean":"2024-04-02T16:22:07.429937+0000","last_became_active":"2024-04-02T16:22:00.390250+0000","last_became_peered":"2024-04-02T16:22:00.390250+0000","last_unstale":"2024-04-02T16:22:07.429937+0000","last_undegraded":"2024-04-02T16:22:07.429937+0000","last_fullsized":"2024-04-02T16:22:07.429937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434997+0000","last_change":"2024-04-02T16:22:05.448031+0000","last_active":"2024-04-02T16:22:07.434997+0000","last_peered":"2024-04-02T16:22:07.434997+0000","last_clean":"2024-04-02T16:22:07.434997+0000","last_became_active":"2024-04-02T16:22:05.447648+0000","last_became_peered":"2024-04-02T16:22:05.447648+0000","last_unstale":"2024-04-02T16:22:07.434997+0000","last_undegraded":"2024-04-02T16:22:07.434997+0000","last_fullsized":"2024-04-02T16:22:07.434997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433811+0000","last_change":"2024-04-02T16:22:01.406024+0000","last_active":"2024-04-02T16:22:07.433811+0000","last_peered":"2024-04-02T16:22:07.433811+0000","last_clean":"2024-04-02T16:22:07.433811+0000","last_became_active":"2024-04-02T16:22:01.405628+0000","last_became_peered":"2024-04-02T16:22:01.405628+0000","last_unstale":"2024-04-02T16:22:07.433811+0000","last_undegraded":"2024-04-02T16:22:07.433811+0000","last_fullsized":"2024-04-02T16:22:07.433811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433971+0000","last_change":"2024-04-02T16:22:00.384113+0000","last_active":"2024-04-02T16:22:07.433971+0000","last_peered":"2024-04-02T16:22:07.433971+0000","last_clean":"2024-04-02T16:22:07.433971+0000","last_became_active":"2024-04-02T16:22:00.383827+0000","last_became_peered":"2024-04-02T16:22:00.383827+0000","last_unstale":"2024-04-02T16:22:07.433971+0000","last_undegraded":"2024-04-02T16:22:07.433971+0000","last_fullsized":"2024-04-02T16:22:07.433971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435258+0000","last_change":"2024-04-02T16:22:05.448007+0000","last_active":"2024-04-02T16:22:07.435258+0000","last_peered":"2024-04-02T16:22:07.435258+0000","last_clean":"2024-04-02T16:22:07.435258+0000","last_became_active":"2024-04-02T16:22:05.447605+0000","last_became_peered":"2024-04-02T16:22:05.447605+0000","last_unstale":"2024-04-02T16:22:07.435258+0000","last_undegraded":"2024-04-02T16:22:07.435258+0000","last_fullsized":"2024-04-02T16:22:07.435258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434278+0000","last_change":"2024-04-02T16:22:01.394732+0000","last_active":"2024-04-02T16:22:07.434278+0000","last_peered":"2024-04-02T16:22:07.434278+0000","last_clean":"2024-04-02T16:22:07.434278+0000","last_became_active":"2024-04-02T16:22:01.394437+0000","last_became_peered":"2024-04-02T16:22:01.394437+0000","last_unstale":"2024-04-02T16:22:07.434278+0000","last_undegraded":"2024-04-02T16:22:07.434278+0000","last_fullsized":"2024-04-02T16:22:07.434278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:26:16.963906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434233+0000","last_change":"2024-04-02T16:22:00.391466+0000","last_active":"2024-04-02T16:22:07.434233+0000","last_peered":"2024-04-02T16:22:07.434233+0000","last_clean":"2024-04-02T16:22:07.434233+0000","last_became_active":"2024-04-02T16:22:00.391148+0000","last_became_peered":"2024-04-02T16:22:00.391148+0000","last_unstale":"2024-04-02T16:22:07.434233+0000","last_undegraded":"2024-04-02T16:22:07.434233+0000","last_fullsized":"2024-04-02T16:22:07.434233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434509+0000","last_change":"2024-04-02T16:22:05.811459+0000","last_active":"2024-04-02T16:22:07.434509+0000","last_peered":"2024-04-02T16:22:07.434509+0000","last_clean":"2024-04-02T16:22:07.434509+0000","last_became_active":"2024-04-02T16:22:05.811036+0000","last_became_peered":"2024-04-02T16:22:05.811036+0000","last_unstale":"2024-04-02T16:22:07.434509+0000","last_undegraded":"2024-04-02T16:22:07.434509+0000","last_fullsized":"2024-04-02T16:22:07.434509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428029+0000","last_change":"2024-04-02T16:22:01.410992+0000","last_active":"2024-04-02T16:22:07.428029+0000","last_peered":"2024-04-02T16:22:07.428029+0000","last_clean":"2024-04-02T16:22:07.428029+0000","last_became_active":"2024-04-02T16:22:01.410701+0000","last_became_peered":"2024-04-02T16:22:01.410701+0000","last_unstale":"2024-04-02T16:22:07.428029+0000","last_undegraded":"2024-04-02T16:22:07.428029+0000","last_fullsized":"2024-04-02T16:22:07.428029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429896+0000","last_change":"2024-04-02T16:22:00.382563+0000","last_active":"2024-04-02T16:22:07.429896+0000","last_peered":"2024-04-02T16:22:07.429896+0000","last_clean":"2024-04-02T16:22:07.429896+0000","last_became_active":"2024-04-02T16:22:00.382285+0000","last_became_peered":"2024-04-02T16:22:00.382285+0000","last_unstale":"2024-04-02T16:22:07.429896+0000","last_undegraded":"2024-04-02T16:22:07.429896+0000","last_fullsized":"2024-04-02T16:22:07.429896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:21:49.851609+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:21:49.850564+0000","last_became_peered":"2024-04-02T16:21:49.850564+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_clean_scrub_stamp":"2024-04-02T16:21:48.692321+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.825754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434765+0000","last_change":"2024-04-02T16:22:05.816624+0000","last_active":"2024-04-02T16:22:07.434765+0000","last_peered":"2024-04-02T16:22:07.434765+0000","last_clean":"2024-04-02T16:22:07.434765+0000","last_became_active":"2024-04-02T16:22:05.816230+0000","last_became_peered":"2024-04-02T16:22:05.816230+0000","last_unstale":"2024-04-02T16:22:07.434765+0000","last_undegraded":"2024-04-02T16:22:07.434765+0000","last_fullsized":"2024-04-02T16:22:07.434765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434871+0000","last_change":"2024-04-02T16:22:01.412438+0000","last_active":"2024-04-02T16:22:07.434871+0000","last_peered":"2024-04-02T16:22:07.434871+0000","last_clean":"2024-04-02T16:22:07.434871+0000","last_became_active":"2024-04-02T16:22:01.411845+0000","last_became_peered":"2024-04-02T16:22:01.411845+0000","last_unstale":"2024-04-02T16:22:07.434871+0000","last_undegraded":"2024-04-02T16:22:07.434871+0000","last_fullsized":"2024-04-02T16:22:07.434871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:22:00.381230+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:22:00.380907+0000","last_became_peered":"2024-04-02T16:22:00.380907+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434320+0000","last_change":"2024-04-02T16:22:05.815548+0000","last_active":"2024-04-02T16:22:07.434320+0000","last_peered":"2024-04-02T16:22:07.434320+0000","last_clean":"2024-04-02T16:22:07.434320+0000","last_became_active":"2024-04-02T16:22:05.815073+0000","last_became_peered":"2024-04-02T16:22:05.815073+0000","last_unstale":"2024-04-02T16:22:07.434320+0000","last_undegraded":"2024-04-02T16:22:07.434320+0000","last_fullsized":"2024-04-02T16:22:07.434320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430426+0000","last_change":"2024-04-02T16:22:01.407537+0000","last_active":"2024-04-02T16:22:07.430426+0000","last_peered":"2024-04-02T16:22:07.430426+0000","last_clean":"2024-04-02T16:22:07.430426+0000","last_became_active":"2024-04-02T16:22:01.403892+0000","last_became_peered":"2024-04-02T16:22:01.403892+0000","last_unstale":"2024-04-02T16:22:07.430426+0000","last_undegraded":"2024-04-02T16:22:07.430426+0000","last_fullsized":"2024-04-02T16:22:07.430426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434911+0000","last_change":"2024-04-02T16:22:00.387875+0000","last_active":"2024-04-02T16:22:07.434911+0000","last_peered":"2024-04-02T16:22:07.434911+0000","last_clean":"2024-04-02T16:22:07.434911+0000","last_became_active":"2024-04-02T16:22:00.387562+0000","last_became_peered":"2024-04-02T16:22:00.387562+0000","last_unstale":"2024-04-02T16:22:07.434911+0000","last_undegraded":"2024-04-02T16:22:07.434911+0000","last_fullsized":"2024-04-02T16:22:07.434911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435508+0000","last_change":"2024-04-02T16:22:05.450999+0000","last_active":"2024-04-02T16:22:07.435508+0000","last_peered":"2024-04-02T16:22:07.435508+0000","last_clean":"2024-04-02T16:22:07.435508+0000","last_became_active":"2024-04-02T16:22:05.450407+0000","last_became_peered":"2024-04-02T16:22:05.450407+0000","last_unstale":"2024-04-02T16:22:07.435508+0000","last_undegraded":"2024-04-02T16:22:07.435508+0000","last_fullsized":"2024-04-02T16:22:07.435508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435483+0000","last_change":"2024-04-02T16:22:01.410592+0000","last_active":"2024-04-02T16:22:07.435483+0000","last_peered":"2024-04-02T16:22:07.435483+0000","last_clean":"2024-04-02T16:22:07.435483+0000","last_became_active":"2024-04-02T16:22:01.410288+0000","last_became_peered":"2024-04-02T16:22:01.410288+0000","last_unstale":"2024-04-02T16:22:07.435483+0000","last_undegraded":"2024-04-02T16:22:07.435483+0000","last_fullsized":"2024-04-02T16:22:07.435483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:58.411440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434019+0000","last_change":"2024-04-02T16:22:00.389059+0000","last_active":"2024-04-02T16:22:07.434019+0000","last_peered":"2024-04-02T16:22:07.434019+0000","last_clean":"2024-04-02T16:22:07.434019+0000","last_became_active":"2024-04-02T16:22:00.388780+0000","last_became_peered":"2024-04-02T16:22:00.388780+0000","last_unstale":"2024-04-02T16:22:07.434019+0000","last_undegraded":"2024-04-02T16:22:07.434019+0000","last_fullsized":"2024-04-02T16:22:07.434019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430338+0000","last_change":"2024-04-02T16:22:05.811513+0000","last_active":"2024-04-02T16:22:07.430338+0000","last_peered":"2024-04-02T16:22:07.430338+0000","last_clean":"2024-04-02T16:22:07.430338+0000","last_became_active":"2024-04-02T16:22:05.811011+0000","last_became_peered":"2024-04-02T16:22:05.811011+0000","last_unstale":"2024-04-02T16:22:07.430338+0000","last_undegraded":"2024-04-02T16:22:07.430338+0000","last_fullsized":"2024-04-02T16:22:07.430338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435147+0000","last_change":"2024-04-02T16:22:01.404383+0000","last_active":"2024-04-02T16:22:07.435147+0000","last_peered":"2024-04-02T16:22:07.435147+0000","last_clean":"2024-04-02T16:22:07.435147+0000","last_became_active":"2024-04-02T16:22:01.404105+0000","last_became_peered":"2024-04-02T16:22:01.404105+0000","last_unstale":"2024-04-02T16:22:07.435147+0000","last_undegraded":"2024-04-02T16:22:07.435147+0000","last_fullsized":"2024-04-02T16:22:07.435147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429344+0000","last_change":"2024-04-02T16:22:00.385712+0000","last_active":"2024-04-02T16:22:07.429344+0000","last_peered":"2024-04-02T16:22:07.429344+0000","last_clean":"2024-04-02T16:22:07.429344+0000","last_became_active":"2024-04-02T16:22:00.385374+0000","last_became_peered":"2024-04-02T16:22:00.385374+0000","last_unstale":"2024-04-02T16:22:07.429344+0000","last_undegraded":"2024-04-02T16:22:07.429344+0000","last_fullsized":"2024-04-02T16:22:07.429344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430015+0000","last_change":"2024-04-02T16:22:05.439393+0000","last_active":"2024-04-02T16:22:07.430015+0000","last_peered":"2024-04-02T16:22:07.430015+0000","last_clean":"2024-04-02T16:22:07.430015+0000","last_became_active":"2024-04-02T16:22:05.439022+0000","last_became_peered":"2024-04-02T16:22:05.439022+0000","last_unstale":"2024-04-02T16:22:07.430015+0000","last_undegraded":"2024-04-02T16:22:07.430015+0000","last_fullsized":"2024-04-02T16:22:07.430015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434659+0000","last_change":"2024-04-02T16:22:01.412452+0000","last_active":"2024-04-02T16:22:07.434659+0000","last_peered":"2024-04-02T16:22:07.434659+0000","last_clean":"2024-04-02T16:22:07.434659+0000","last_became_active":"2024-04-02T16:22:01.412160+0000","last_became_peered":"2024-04-02T16:22:01.412160+0000","last_unstale":"2024-04-02T16:22:07.434659+0000","last_undegraded":"2024-04-02T16:22:07.434659+0000","last_fullsized":"2024-04-02T16:22:07.434659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433532+0000","last_change":"2024-04-02T16:22:00.386167+0000","last_active":"2024-04-02T16:22:07.433532+0000","last_peered":"2024-04-02T16:22:07.433532+0000","last_clean":"2024-04-02T16:22:07.433532+0000","last_became_active":"2024-04-02T16:22:00.385887+0000","last_became_peered":"2024-04-02T16:22:00.385887+0000","last_unstale":"2024-04-02T16:22:07.433532+0000","last_undegraded":"2024-04-02T16:22:07.433532+0000","last_fullsized":"2024-04-02T16:22:07.433532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434360+0000","last_change":"2024-04-02T16:22:05.451144+0000","last_active":"2024-04-02T16:22:07.434360+0000","last_peered":"2024-04-02T16:22:07.434360+0000","last_clean":"2024-04-02T16:22:07.434360+0000","last_became_active":"2024-04-02T16:22:05.450690+0000","last_became_peered":"2024-04-02T16:22:05.450690+0000","last_unstale":"2024-04-02T16:22:07.434360+0000","last_undegraded":"2024-04-02T16:22:07.434360+0000","last_fullsized":"2024-04-02T16:22:07.434360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434308+0000","last_change":"2024-04-02T16:22:01.409326+0000","last_active":"2024-04-02T16:22:07.434308+0000","last_peered":"2024-04-02T16:22:07.434308+0000","last_clean":"2024-04-02T16:22:07.434308+0000","last_became_active":"2024-04-02T16:22:01.408710+0000","last_became_peered":"2024-04-02T16:22:01.408710+0000","last_unstale":"2024-04-02T16:22:07.434308+0000","last_undegraded":"2024-04-02T16:22:07.434308+0000","last_fullsized":"2024-04-02T16:22:07.434308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434473+0000","last_change":"2024-04-02T16:22:00.389406+0000","last_active":"2024-04-02T16:22:07.434473+0000","last_peered":"2024-04-02T16:22:07.434473+0000","last_clean":"2024-04-02T16:22:07.434473+0000","last_became_active":"2024-04-02T16:22:00.389107+0000","last_became_peered":"2024-04-02T16:22:00.389107+0000","last_unstale":"2024-04-02T16:22:07.434473+0000","last_undegraded":"2024-04-02T16:22:07.434473+0000","last_fullsized":"2024-04-02T16:22:07.434473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:05.816005+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:45.290851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434745+0000","last_change":"2024-04-02T16:22:01.412447+0000","last_active":"2024-04-02T16:22:07.434745+0000","last_peered":"2024-04-02T16:22:07.434745+0000","last_clean":"2024-04-02T16:22:07.434745+0000","last_became_active":"2024-04-02T16:22:01.411677+0000","last_became_peered":"2024-04-02T16:22:01.411677+0000","last_unstale":"2024-04-02T16:22:07.434745+0000","last_undegraded":"2024-04-02T16:22:07.434745+0000","last_fullsized":"2024-04-02T16:22:07.434745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:00.389887+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:00.389287+0000","last_became_peered":"2024-04-02T16:22:00.389287+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428399+0000","last_change":"2024-04-02T16:22:05.815236+0000","last_active":"2024-04-02T16:22:07.428399+0000","last_peered":"2024-04-02T16:22:07.428399+0000","last_clean":"2024-04-02T16:22:07.428399+0000","last_became_active":"2024-04-02T16:22:05.814782+0000","last_became_peered":"2024-04-02T16:22:05.814782+0000","last_unstale":"2024-04-02T16:22:07.428399+0000","last_undegraded":"2024-04-02T16:22:07.428399+0000","last_fullsized":"2024-04-02T16:22:07.428399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:44.247231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434209+0000","last_change":"2024-04-02T16:22:01.401164+0000","last_active":"2024-04-02T16:22:07.434209+0000","last_peered":"2024-04-02T16:22:07.434209+0000","last_clean":"2024-04-02T16:22:07.434209+0000","last_became_active":"2024-04-02T16:22:01.400880+0000","last_became_peered":"2024-04-02T16:22:01.400880+0000","last_unstale":"2024-04-02T16:22:07.434209+0000","last_undegraded":"2024-04-02T16:22:07.434209+0000","last_fullsized":"2024-04-02T16:22:07.434209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435570+0000","last_change":"2024-04-02T16:22:00.386660+0000","last_active":"2024-04-02T16:22:07.435570+0000","last_peered":"2024-04-02T16:22:07.435570+0000","last_clean":"2024-04-02T16:22:07.435570+0000","last_became_active":"2024-04-02T16:22:00.386370+0000","last_became_peered":"2024-04-02T16:22:00.386370+0000","last_unstale":"2024-04-02T16:22:07.435570+0000","last_undegraded":"2024-04-02T16:22:07.435570+0000","last_fullsized":"2024-04-02T16:22:07.435570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430760+0000","last_change":"2024-04-02T16:22:05.816956+0000","last_active":"2024-04-02T16:22:07.430760+0000","last_peered":"2024-04-02T16:22:07.430760+0000","last_clean":"2024-04-02T16:22:07.430760+0000","last_became_active":"2024-04-02T16:22:05.816429+0000","last_became_peered":"2024-04-02T16:22:05.816429+0000","last_unstale":"2024-04-02T16:22:07.430760+0000","last_undegraded":"2024-04-02T16:22:07.430760+0000","last_fullsized":"2024-04-02T16:22:07.430760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434384+0000","last_change":"2024-04-02T16:22:01.405624+0000","last_active":"2024-04-02T16:22:07.434384+0000","last_peered":"2024-04-02T16:22:07.434384+0000","last_clean":"2024-04-02T16:22:07.434384+0000","last_became_active":"2024-04-02T16:22:01.405347+0000","last_became_peered":"2024-04-02T16:22:01.405347+0000","last_unstale":"2024-04-02T16:22:07.434384+0000","last_undegraded":"2024-04-02T16:22:07.434384+0000","last_fullsized":"2024-04-02T16:22:07.434384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:10.942808+0000","last_change":"2024-04-02T16:22:00.391986+0000","last_active":"2024-04-02T16:22:10.942808+0000","last_peered":"2024-04-02T16:22:10.942808+0000","last_clean":"2024-04-02T16:22:10.942808+0000","last_became_active":"2024-04-02T16:22:00.391681+0000","last_became_peered":"2024-04-02T16:22:00.391681+0000","last_unstale":"2024-04-02T16:22:10.942808+0000","last_undegraded":"2024-04-02T16:22:10.942808+0000","last_fullsized":"2024-04-02T16:22:10.942808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434848+0000","last_change":"2024-04-02T16:22:05.446704+0000","last_active":"2024-04-02T16:22:07.434848+0000","last_peered":"2024-04-02T16:22:07.434848+0000","last_clean":"2024-04-02T16:22:07.434848+0000","last_became_active":"2024-04-02T16:22:05.446150+0000","last_became_peered":"2024-04-02T16:22:05.446150+0000","last_unstale":"2024-04-02T16:22:07.434848+0000","last_undegraded":"2024-04-02T16:22:07.434848+0000","last_fullsized":"2024-04-02T16:22:07.434848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:01.400842+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:01.400565+0000","last_became_peered":"2024-04-02T16:22:01.400565+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434908+0000","last_change":"2024-04-02T16:22:00.394135+0000","last_active":"2024-04-02T16:22:07.434908+0000","last_peered":"2024-04-02T16:22:07.434908+0000","last_clean":"2024-04-02T16:22:07.434908+0000","last_became_active":"2024-04-02T16:22:00.393846+0000","last_became_peered":"2024-04-02T16:22:00.393846+0000","last_unstale":"2024-04-02T16:22:07.434908+0000","last_undegraded":"2024-04-02T16:22:07.434908+0000","last_fullsized":"2024-04-02T16:22:07.434908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429907+0000","last_change":"2024-04-02T16:22:05.439118+0000","last_active":"2024-04-02T16:22:07.429907+0000","last_peered":"2024-04-02T16:22:07.429907+0000","last_clean":"2024-04-02T16:22:07.429907+0000","last_became_active":"2024-04-02T16:22:05.438557+0000","last_became_peered":"2024-04-02T16:22:05.438557+0000","last_unstale":"2024-04-02T16:22:07.429907+0000","last_undegraded":"2024-04-02T16:22:07.429907+0000","last_fullsized":"2024-04-02T16:22:07.429907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434885+0000","last_change":"2024-04-02T16:22:01.413531+0000","last_active":"2024-04-02T16:22:07.434885+0000","last_peered":"2024-04-02T16:22:07.434885+0000","last_clean":"2024-04-02T16:22:07.434885+0000","last_became_active":"2024-04-02T16:22:01.413229+0000","last_became_peered":"2024-04-02T16:22:01.413229+0000","last_unstale":"2024-04-02T16:22:07.434885+0000","last_undegraded":"2024-04-02T16:22:07.434885+0000","last_fullsized":"2024-04-02T16:22:07.434885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430736+0000","last_change":"2024-04-02T16:22:00.388772+0000","last_active":"2024-04-02T16:22:07.430736+0000","last_peered":"2024-04-02T16:22:07.430736+0000","last_clean":"2024-04-02T16:22:07.430736+0000","last_became_active":"2024-04-02T16:22:00.388500+0000","last_became_peered":"2024-04-02T16:22:00.388500+0000","last_unstale":"2024-04-02T16:22:07.430736+0000","last_undegraded":"2024-04-02T16:22:07.430736+0000","last_fullsized":"2024-04-02T16:22:07.430736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427878+0000","last_change":"2024-04-02T16:22:05.441542+0000","last_active":"2024-04-02T16:22:07.427878+0000","last_peered":"2024-04-02T16:22:07.427878+0000","last_clean":"2024-04-02T16:22:07.427878+0000","last_became_active":"2024-04-02T16:22:05.441134+0000","last_became_peered":"2024-04-02T16:22:05.441134+0000","last_unstale":"2024-04-02T16:22:07.427878+0000","last_undegraded":"2024-04-02T16:22:07.427878+0000","last_fullsized":"2024-04-02T16:22:07.427878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:02:44.237629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430701+0000","last_change":"2024-04-02T16:22:01.401315+0000","last_active":"2024-04-02T16:22:07.430701+0000","last_peered":"2024-04-02T16:22:07.430701+0000","last_clean":"2024-04-02T16:22:07.430701+0000","last_became_active":"2024-04-02T16:22:01.401023+0000","last_became_peered":"2024-04-02T16:22:01.401023+0000","last_unstale":"2024-04-02T16:22:07.430701+0000","last_undegraded":"2024-04-02T16:22:07.430701+0000","last_fullsized":"2024-04-02T16:22:07.430701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435375+0000","last_change":"2024-04-02T16:22:00.394082+0000","last_active":"2024-04-02T16:22:07.435375+0000","last_peered":"2024-04-02T16:22:07.435375+0000","last_clean":"2024-04-02T16:22:07.435375+0000","last_became_active":"2024-04-02T16:22:00.393793+0000","last_became_peered":"2024-04-02T16:22:00.393793+0000","last_unstale":"2024-04-02T16:22:07.435375+0000","last_undegraded":"2024-04-02T16:22:07.435375+0000","last_fullsized":"2024-04-02T16:22:07.435375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427851+0000","last_change":"2024-04-02T16:22:05.817432+0000","last_active":"2024-04-02T16:22:07.427851+0000","last_peered":"2024-04-02T16:22:07.427851+0000","last_clean":"2024-04-02T16:22:07.427851+0000","last_became_active":"2024-04-02T16:22:05.817018+0000","last_became_peered":"2024-04-02T16:22:05.817018+0000","last_unstale":"2024-04-02T16:22:07.427851+0000","last_undegraded":"2024-04-02T16:22:07.427851+0000","last_fullsized":"2024-04-02T16:22:07.427851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:52:22.886852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434205+0000","last_change":"2024-04-02T16:22:05.815342+0000","last_active":"2024-04-02T16:22:07.434205+0000","last_peered":"2024-04-02T16:22:07.434205+0000","last_clean":"2024-04-02T16:22:07.434205+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.434205+0000","last_undegraded":"2024-04-02T16:22:07.434205+0000","last_fullsized":"2024-04-02T16:22:07.434205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434398+0000","last_change":"2024-04-02T16:22:01.400394+0000","last_active":"2024-04-02T16:22:07.434398+0000","last_peered":"2024-04-02T16:22:07.434398+0000","last_clean":"2024-04-02T16:22:07.434398+0000","last_became_active":"2024-04-02T16:22:01.400000+0000","last_became_peered":"2024-04-02T16:22:01.400000+0000","last_unstale":"2024-04-02T16:22:07.434398+0000","last_undegraded":"2024-04-02T16:22:07.434398+0000","last_fullsized":"2024-04-02T16:22:07.434398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430602+0000","last_change":"2024-04-02T16:22:00.380328+0000","last_active":"2024-04-02T16:22:07.430602+0000","last_peered":"2024-04-02T16:22:07.430602+0000","last_clean":"2024-04-02T16:22:07.430602+0000","last_became_active":"2024-04-02T16:22:00.379294+0000","last_became_peered":"2024-04-02T16:22:00.379294+0000","last_unstale":"2024-04-02T16:22:07.430602+0000","last_undegraded":"2024-04-02T16:22:07.430602+0000","last_fullsized":"2024-04-02T16:22:07.430602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435187+0000","last_change":"2024-04-02T16:22:05.444408+0000","last_active":"2024-04-02T16:22:07.435187+0000","last_peered":"2024-04-02T16:22:07.435187+0000","last_clean":"2024-04-02T16:22:07.435187+0000","last_became_active":"2024-04-02T16:22:05.444100+0000","last_became_peered":"2024-04-02T16:22:05.444100+0000","last_unstale":"2024-04-02T16:22:07.435187+0000","last_undegraded":"2024-04-02T16:22:07.435187+0000","last_fullsized":"2024-04-02T16:22:07.435187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427872+0000","last_change":"2024-04-02T16:22:01.409456+0000","last_active":"2024-04-02T16:22:07.427872+0000","last_peered":"2024-04-02T16:22:07.427872+0000","last_clean":"2024-04-02T16:22:07.427872+0000","last_became_active":"2024-04-02T16:22:01.408824+0000","last_became_peered":"2024-04-02T16:22:01.408824+0000","last_unstale":"2024-04-02T16:22:07.427872+0000","last_undegraded":"2024-04-02T16:22:07.427872+0000","last_fullsized":"2024-04-02T16:22:07.427872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427970+0000","last_change":"2024-04-02T16:22:00.390487+0000","last_active":"2024-04-02T16:22:07.427970+0000","last_peered":"2024-04-02T16:22:07.427970+0000","last_clean":"2024-04-02T16:22:07.427970+0000","last_became_active":"2024-04-02T16:22:00.390198+0000","last_became_peered":"2024-04-02T16:22:00.390198+0000","last_unstale":"2024-04-02T16:22:07.427970+0000","last_undegraded":"2024-04-02T16:22:07.427970+0000","last_fullsized":"2024-04-02T16:22:07.427970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434683+0000","last_change":"2024-04-02T16:22:00.388624+0000","last_active":"2024-04-02T16:22:07.434683+0000","last_peered":"2024-04-02T16:22:07.434683+0000","last_clean":"2024-04-02T16:22:07.434683+0000","last_became_active":"2024-04-02T16:22:00.388351+0000","last_became_peered":"2024-04-02T16:22:00.388351+0000","last_unstale":"2024-04-02T16:22:07.434683+0000","last_undegraded":"2024-04-02T16:22:07.434683+0000","last_fullsized":"2024-04-02T16:22:07.434683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435511+0000","last_change":"2024-04-02T16:22:01.403982+0000","last_active":"2024-04-02T16:22:07.435511+0000","last_peered":"2024-04-02T16:22:07.435511+0000","last_clean":"2024-04-02T16:22:07.435511+0000","last_became_active":"2024-04-02T16:22:01.403057+0000","last_became_peered":"2024-04-02T16:22:01.403057+0000","last_unstale":"2024-04-02T16:22:07.435511+0000","last_undegraded":"2024-04-02T16:22:07.435511+0000","last_fullsized":"2024-04-02T16:22:07.435511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434167+0000","last_change":"2024-04-02T16:22:05.443373+0000","last_active":"2024-04-02T16:22:07.434167+0000","last_peered":"2024-04-02T16:22:07.434167+0000","last_clean":"2024-04-02T16:22:07.434167+0000","last_became_active":"2024-04-02T16:22:05.442934+0000","last_became_peered":"2024-04-02T16:22:05.442934+0000","last_unstale":"2024-04-02T16:22:07.434167+0000","last_undegraded":"2024-04-02T16:22:07.434167+0000","last_fullsized":"2024-04-02T16:22:07.434167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:49.857086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428264+0000","last_change":"2024-04-02T16:22:00.391680+0000","last_active":"2024-04-02T16:22:07.428264+0000","last_peered":"2024-04-02T16:22:07.428264+0000","last_clean":"2024-04-02T16:22:07.428264+0000","last_became_active":"2024-04-02T16:22:00.391402+0000","last_became_peered":"2024-04-02T16:22:00.391402+0000","last_unstale":"2024-04-02T16:22:07.428264+0000","last_undegraded":"2024-04-02T16:22:07.428264+0000","last_fullsized":"2024-04-02T16:22:07.428264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429224+0000","last_change":"2024-04-02T16:22:01.398824+0000","last_active":"2024-04-02T16:22:07.429224+0000","last_peered":"2024-04-02T16:22:07.429224+0000","last_clean":"2024-04-02T16:22:07.429224+0000","last_became_active":"2024-04-02T16:22:01.398529+0000","last_became_peered":"2024-04-02T16:22:01.398529+0000","last_unstale":"2024-04-02T16:22:07.429224+0000","last_undegraded":"2024-04-02T16:22:07.429224+0000","last_fullsized":"2024-04-02T16:22:07.429224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:01.320427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428686+0000","last_change":"2024-04-02T16:22:05.440423+0000","last_active":"2024-04-02T16:22:07.428686+0000","last_peered":"2024-04-02T16:22:07.428686+0000","last_clean":"2024-04-02T16:22:07.428686+0000","last_became_active":"2024-04-02T16:22:05.439993+0000","last_became_peered":"2024-04-02T16:22:05.439993+0000","last_unstale":"2024-04-02T16:22:07.428686+0000","last_undegraded":"2024-04-02T16:22:07.428686+0000","last_fullsized":"2024-04-02T16:22:07.428686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:17.805386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434177+0000","last_change":"2024-04-02T16:22:01.408902+0000","last_active":"2024-04-02T16:22:07.434177+0000","last_peered":"2024-04-02T16:22:07.434177+0000","last_clean":"2024-04-02T16:22:07.434177+0000","last_became_active":"2024-04-02T16:22:01.408562+0000","last_became_peered":"2024-04-02T16:22:01.408562+0000","last_unstale":"2024-04-02T16:22:07.434177+0000","last_undegraded":"2024-04-02T16:22:07.434177+0000","last_fullsized":"2024-04-02T16:22:07.434177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433873+0000","last_change":"2024-04-02T16:22:01.404001+0000","last_active":"2024-04-02T16:22:07.433873+0000","last_peered":"2024-04-02T16:22:07.433873+0000","last_clean":"2024-04-02T16:22:07.433873+0000","last_became_active":"2024-04-02T16:22:01.403694+0000","last_became_peered":"2024-04-02T16:22:01.403694+0000","last_unstale":"2024-04-02T16:22:07.433873+0000","last_undegraded":"2024-04-02T16:22:07.433873+0000","last_fullsized":"2024-04-02T16:22:07.433873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429709+0000","last_change":"2024-04-02T16:22:01.407141+0000","last_active":"2024-04-02T16:22:07.429709+0000","last_peered":"2024-04-02T16:22:07.429709+0000","last_clean":"2024-04-02T16:22:07.429709+0000","last_became_active":"2024-04-02T16:22:01.403018+0000","last_became_peered":"2024-04-02T16:22:01.403018+0000","last_unstale":"2024-04-02T16:22:07.429709+0000","last_undegraded":"2024-04-02T16:22:07.429709+0000","last_fullsized":"2024-04-02T16:22:07.429709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435416+0000","last_change":"2024-04-02T16:22:01.403712+0000","last_active":"2024-04-02T16:22:07.435416+0000","last_peered":"2024-04-02T16:22:07.435416+0000","last_clean":"2024-04-02T16:22:07.435416+0000","last_became_active":"2024-04-02T16:22:01.402419+0000","last_became_peered":"2024-04-02T16:22:01.402419+0000","last_unstale":"2024-04-02T16:22:07.435416+0000","last_undegraded":"2024-04-02T16:22:07.435416+0000","last_fullsized":"2024-04-02T16:22:07.435416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428475+0000","last_change":"2024-04-02T16:22:01.411196+0000","last_active":"2024-04-02T16:22:07.428475+0000","last_peered":"2024-04-02T16:22:07.428475+0000","last_clean":"2024-04-02T16:22:07.428475+0000","last_became_active":"2024-04-02T16:22:01.410630+0000","last_became_peered":"2024-04-02T16:22:01.410630+0000","last_unstale":"2024-04-02T16:22:07.428475+0000","last_undegraded":"2024-04-02T16:22:07.428475+0000","last_fullsized":"2024-04-02T16:22:07.428475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434802+0000","last_change":"2024-04-02T16:22:01.401113+0000","last_active":"2024-04-02T16:22:07.434802+0000","last_peered":"2024-04-02T16:22:07.434802+0000","last_clean":"2024-04-02T16:22:07.434802+0000","last_became_active":"2024-04-02T16:22:01.400823+0000","last_became_peered":"2024-04-02T16:22:01.400823+0000","last_unstale":"2024-04-02T16:22:07.434802+0000","last_undegraded":"2024-04-02T16:22:07.434802+0000","last_fullsized":"2024-04-02T16:22:07.434802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435248+0000","last_change":"2024-04-02T16:22:01.401273+0000","last_active":"2024-04-02T16:22:07.435248+0000","last_peered":"2024-04-02T16:22:07.435248+0000","last_clean":"2024-04-02T16:22:07.435248+0000","last_became_active":"2024-04-02T16:22:01.400989+0000","last_became_peered":"2024-04-02T16:22:01.400989+0000","last_unstale":"2024-04-02T16:22:07.435248+0000","last_undegraded":"2024-04-02T16:22:07.435248+0000","last_fullsized":"2024-04-02T16:22:07.435248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428640+0000","last_change":"2024-04-02T16:22:01.410167+0000","last_active":"2024-04-02T16:22:07.428640+0000","last_peered":"2024-04-02T16:22:07.428640+0000","last_clean":"2024-04-02T16:22:07.428640+0000","last_became_active":"2024-04-02T16:22:01.409747+0000","last_became_peered":"2024-04-02T16:22:01.409747+0000","last_unstale":"2024-04-02T16:22:07.428640+0000","last_undegraded":"2024-04-02T16:22:07.428640+0000","last_fullsized":"2024-04-02T16:22:07.428640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430485+0000","last_change":"2024-04-02T16:22:01.404722+0000","last_active":"2024-04-02T16:22:07.430485+0000","last_peered":"2024-04-02T16:22:07.430485+0000","last_clean":"2024-04-02T16:22:07.430485+0000","last_became_active":"2024-04-02T16:22:01.403645+0000","last_became_peered":"2024-04-02T16:22:01.403645+0000","last_unstale":"2024-04-02T16:22:07.430485+0000","last_undegraded":"2024-04-02T16:22:07.430485+0000","last_fullsized":"2024-04-02T16:22:07.430485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427877+0000","last_change":"2024-04-02T16:22:01.410096+0000","last_active":"2024-04-02T16:22:07.427877+0000","last_peered":"2024-04-02T16:22:07.427877+0000","last_clean":"2024-04-02T16:22:07.427877+0000","last_became_active":"2024-04-02T16:22:01.409805+0000","last_became_peered":"2024-04-02T16:22:01.409805+0000","last_unstale":"2024-04-02T16:22:07.427877+0000","last_undegraded":"2024-04-02T16:22:07.427877+0000","last_fullsized":"2024-04-02T16:22:07.427877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429582+0000","last_change":"2024-04-02T16:22:01.410138+0000","last_active":"2024-04-02T16:22:07.429582+0000","last_peered":"2024-04-02T16:22:07.429582+0000","last_clean":"2024-04-02T16:22:07.429582+0000","last_became_active":"2024-04-02T16:22:01.403179+0000","last_became_peered":"2024-04-02T16:22:01.403179+0000","last_unstale":"2024-04-02T16:22:07.429582+0000","last_undegraded":"2024-04-02T16:22:07.429582+0000","last_fullsized":"2024-04-02T16:22:07.429582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435300+0000","last_change":"2024-04-02T16:22:01.403457+0000","last_active":"2024-04-02T16:22:07.435300+0000","last_peered":"2024-04-02T16:22:07.435300+0000","last_clean":"2024-04-02T16:22:07.435300+0000","last_became_active":"2024-04-02T16:22:01.402335+0000","last_became_peered":"2024-04-02T16:22:01.402335+0000","last_unstale":"2024-04-02T16:22:07.435300+0000","last_undegraded":"2024-04-02T16:22:07.435300+0000","last_fullsized":"2024-04-02T16:22:07.435300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434948+0000","last_change":"2024-04-02T16:22:01.406544+0000","last_active":"2024-04-02T16:22:07.434948+0000","last_peered":"2024-04-02T16:22:07.434948+0000","last_clean":"2024-04-02T16:22:07.434948+0000","last_became_active":"2024-04-02T16:22:01.406205+0000","last_became_peered":"2024-04-02T16:22:01.406205+0000","last_unstale":"2024-04-02T16:22:07.434948+0000","last_undegraded":"2024-04-02T16:22:07.434948+0000","last_fullsized":"2024-04-02T16:22:07.434948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435085+0000","last_change":"2024-04-02T16:22:01.401972+0000","last_active":"2024-04-02T16:22:07.435085+0000","last_peered":"2024-04-02T16:22:07.435085+0000","last_clean":"2024-04-02T16:22:07.435085+0000","last_became_active":"2024-04-02T16:22:01.401721+0000","last_became_peered":"2024-04-02T16:22:01.401721+0000","last_unstale":"2024-04-02T16:22:07.435085+0000","last_undegraded":"2024-04-02T16:22:07.435085+0000","last_fullsized":"2024-04-02T16:22:07.435085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:05.479052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433925+0000","last_change":"2024-04-02T16:22:01.406917+0000","last_active":"2024-04-02T16:22:07.433925+0000","last_peered":"2024-04-02T16:22:07.433925+0000","last_clean":"2024-04-02T16:22:07.433925+0000","last_became_active":"2024-04-02T16:22:01.406622+0000","last_became_peered":"2024-04-02T16:22:01.406622+0000","last_unstale":"2024-04-02T16:22:07.433925+0000","last_undegraded":"2024-04-02T16:22:07.433925+0000","last_fullsized":"2024-04-02T16:22:07.433925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430970+0000","last_change":"2024-04-02T16:22:01.403891+0000","last_active":"2024-04-02T16:22:07.430970+0000","last_peered":"2024-04-02T16:22:07.430970+0000","last_clean":"2024-04-02T16:22:07.430970+0000","last_became_active":"2024-04-02T16:22:01.403250+0000","last_became_peered":"2024-04-02T16:22:01.403250+0000","last_unstale":"2024-04-02T16:22:07.430970+0000","last_undegraded":"2024-04-02T16:22:07.430970+0000","last_fullsized":"2024-04-02T16:22:07.430970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430459+0000","last_change":"2024-04-02T16:22:01.410686+0000","last_active":"2024-04-02T16:22:07.430459+0000","last_peered":"2024-04-02T16:22:07.430459+0000","last_clean":"2024-04-02T16:22:07.430459+0000","last_became_active":"2024-04-02T16:22:01.410398+0000","last_became_peered":"2024-04-02T16:22:01.410398+0000","last_unstale":"2024-04-02T16:22:07.430459+0000","last_undegraded":"2024-04-02T16:22:07.430459+0000","last_fullsized":"2024-04-02T16:22:07.430459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.406912+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.406626+0000","last_became_peered":"2024-04-02T16:22:01.406626+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434095+0000","last_change":"2024-04-02T16:22:01.397661+0000","last_active":"2024-04-02T16:22:07.434095+0000","last_peered":"2024-04-02T16:22:07.434095+0000","last_clean":"2024-04-02T16:22:07.434095+0000","last_became_active":"2024-04-02T16:22:01.397335+0000","last_became_peered":"2024-04-02T16:22:01.397335+0000","last_unstale":"2024-04-02T16:22:07.434095+0000","last_undegraded":"2024-04-02T16:22:07.434095+0000","last_fullsized":"2024-04-02T16:22:07.434095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428147+0000","last_change":"2024-04-02T16:22:01.409121+0000","last_active":"2024-04-02T16:22:07.428147+0000","last_peered":"2024-04-02T16:22:07.428147+0000","last_clean":"2024-04-02T16:22:07.428147+0000","last_became_active":"2024-04-02T16:22:01.408765+0000","last_became_peered":"2024-04-02T16:22:01.408765+0000","last_unstale":"2024-04-02T16:22:07.428147+0000","last_undegraded":"2024-04-02T16:22:07.428147+0000","last_fullsized":"2024-04-02T16:22:07.428147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429788+0000","last_change":"2024-04-02T16:22:01.407108+0000","last_active":"2024-04-02T16:22:07.429788+0000","last_peered":"2024-04-02T16:22:07.429788+0000","last_clean":"2024-04-02T16:22:07.429788+0000","last_became_active":"2024-04-02T16:22:01.406836+0000","last_became_peered":"2024-04-02T16:22:01.406836+0000","last_unstale":"2024-04-02T16:22:07.429788+0000","last_undegraded":"2024-04-02T16:22:07.429788+0000","last_fullsized":"2024-04-02T16:22:07.429788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434905+0000","last_change":"2024-04-02T16:22:01.414174+0000","last_active":"2024-04-02T16:22:07.434905+0000","last_peered":"2024-04-02T16:22:07.434905+0000","last_clean":"2024-04-02T16:22:07.434905+0000","last_became_active":"2024-04-02T16:22:01.413903+0000","last_became_peered":"2024-04-02T16:22:01.413903+0000","last_unstale":"2024-04-02T16:22:07.434905+0000","last_undegraded":"2024-04-02T16:22:07.434905+0000","last_fullsized":"2024-04-02T16:22:07.434905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435234+0000","last_change":"2024-04-02T16:22:01.413533+0000","last_active":"2024-04-02T16:22:07.435234+0000","last_peered":"2024-04-02T16:22:07.435234+0000","last_clean":"2024-04-02T16:22:07.435234+0000","last_became_active":"2024-04-02T16:22:01.413266+0000","last_became_peered":"2024-04-02T16:22:01.413266+0000","last_unstale":"2024-04-02T16:22:07.435234+0000","last_undegraded":"2024-04-02T16:22:07.435234+0000","last_fullsized":"2024-04-02T16:22:07.435234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435468+0000","last_change":"2024-04-02T16:22:01.412176+0000","last_active":"2024-04-02T16:22:07.435468+0000","last_peered":"2024-04-02T16:22:07.435468+0000","last_clean":"2024-04-02T16:22:07.435468+0000","last_became_active":"2024-04-02T16:22:01.411793+0000","last_became_peered":"2024-04-02T16:22:01.411793+0000","last_unstale":"2024-04-02T16:22:07.435468+0000","last_undegraded":"2024-04-02T16:22:07.435468+0000","last_fullsized":"2024-04-02T16:22:07.435468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:12.015838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427771+0000","last_change":"2024-04-02T16:22:01.410605+0000","last_active":"2024-04-02T16:22:07.427771+0000","last_peered":"2024-04-02T16:22:07.427771+0000","last_clean":"2024-04-02T16:22:07.427771+0000","last_became_active":"2024-04-02T16:22:01.410237+0000","last_became_peered":"2024-04-02T16:22:01.410237+0000","last_unstale":"2024-04-02T16:22:07.427771+0000","last_undegraded":"2024-04-02T16:22:07.427771+0000","last_fullsized":"2024-04-02T16:22:07.427771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434564+0000","last_change":"2024-04-02T16:22:01.404165+0000","last_active":"2024-04-02T16:22:07.434564+0000","last_peered":"2024-04-02T16:22:07.434564+0000","last_clean":"2024-04-02T16:22:07.434564+0000","last_became_active":"2024-04-02T16:22:01.403907+0000","last_became_peered":"2024-04-02T16:22:01.403907+0000","last_unstale":"2024-04-02T16:22:07.434564+0000","last_undegraded":"2024-04-02T16:22:07.434564+0000","last_fullsized":"2024-04-02T16:22:07.434564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429480+0000","last_change":"2024-04-02T16:22:01.409810+0000","last_active":"2024-04-02T16:22:07.429480+0000","last_peered":"2024-04-02T16:22:07.429480+0000","last_clean":"2024-04-02T16:22:07.429480+0000","last_became_active":"2024-04-02T16:22:01.406447+0000","last_became_peered":"2024-04-02T16:22:01.406447+0000","last_unstale":"2024-04-02T16:22:07.429480+0000","last_undegraded":"2024-04-02T16:22:07.429480+0000","last_fullsized":"2024-04-02T16:22:07.429480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434648+0000","last_change":"2024-04-02T16:22:01.401898+0000","last_active":"2024-04-02T16:22:07.434648+0000","last_peered":"2024-04-02T16:22:07.434648+0000","last_clean":"2024-04-02T16:22:07.434648+0000","last_became_active":"2024-04-02T16:22:01.401568+0000","last_became_peered":"2024-04-02T16:22:01.401568+0000","last_unstale":"2024-04-02T16:22:07.434648+0000","last_undegraded":"2024-04-02T16:22:07.434648+0000","last_fullsized":"2024-04-02T16:22:07.434648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430221+0000","last_change":"2024-04-02T16:22:01.410551+0000","last_active":"2024-04-02T16:22:07.430221+0000","last_peered":"2024-04-02T16:22:07.430221+0000","last_clean":"2024-04-02T16:22:07.430221+0000","last_became_active":"2024-04-02T16:22:01.409045+0000","last_became_peered":"2024-04-02T16:22:01.409045+0000","last_unstale":"2024-04-02T16:22:07.430221+0000","last_undegraded":"2024-04-02T16:22:07.430221+0000","last_fullsized":"2024-04-02T16:22:07.430221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434517+0000","last_change":"2024-04-02T16:22:01.398994+0000","last_active":"2024-04-02T16:22:07.434517+0000","last_peered":"2024-04-02T16:22:07.434517+0000","last_clean":"2024-04-02T16:22:07.434517+0000","last_became_active":"2024-04-02T16:22:01.398698+0000","last_became_peered":"2024-04-02T16:22:01.398698+0000","last_unstale":"2024-04-02T16:22:07.434517+0000","last_undegraded":"2024-04-02T16:22:07.434517+0000","last_fullsized":"2024-04-02T16:22:07.434517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435172+0000","last_change":"2024-04-02T16:22:01.406275+0000","last_active":"2024-04-02T16:22:07.435172+0000","last_peered":"2024-04-02T16:22:07.435172+0000","last_clean":"2024-04-02T16:22:07.435172+0000","last_became_active":"2024-04-02T16:22:01.405936+0000","last_became_peered":"2024-04-02T16:22:01.405936+0000","last_unstale":"2024-04-02T16:22:07.435172+0000","last_undegraded":"2024-04-02T16:22:07.435172+0000","last_fullsized":"2024-04-02T16:22:07.435172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435388+0000","last_change":"2024-04-02T16:22:01.407470+0000","last_active":"2024-04-02T16:22:07.435388+0000","last_peered":"2024-04-02T16:22:07.435388+0000","last_clean":"2024-04-02T16:22:07.435388+0000","last_became_active":"2024-04-02T16:22:01.407209+0000","last_became_peered":"2024-04-02T16:22:01.407209+0000","last_unstale":"2024-04-02T16:22:07.435388+0000","last_undegraded":"2024-04-02T16:22:07.435388+0000","last_fullsized":"2024-04-02T16:22:07.435388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:31.303906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:22:28.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:22:29.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.096+0000 7fe7551e4640 1 Processor -- start 2024-04-02T16:22:29.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.096+0000 7fe7551e4640 1 -- start start 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe7551e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75006bcf0 0x7fe7500ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe7551e4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe7500b0690 0x7fe7500addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe7551e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7500b4bb0 con 0x7fe750161b80 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe7551e4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fe750072320 con 0x7fe7500b0690 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe7551e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe750072570 con 0x7fe75006bcf0 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe74f59e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe7500b0690 0x7fe7500addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe74f59e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe7500b0690 0x7fe7500addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45362/0 (socket says 172.21.15.50:45362) 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe74f59e640 1 -- 172.21.15.50:0/3052842638 learned_addr learned my addr 172.21.15.50:0/3052842638 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe747fff640 1 --2- 172.21.15.50:0/3052842638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75006bcf0 0x7fe7500ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe74f59e640 1 -- 172.21.15.50:0/3052842638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75006bcf0 msgr2=0x7fe7500ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe74f59e640 1 --2- 172.21.15.50:0/3052842638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75006bcf0 0x7fe7500ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe74f59e640 1 -- 172.21.15.50:0/3052842638 >> v1:172.21.15.188:6789/0 conn(0x7fe750161b80 legacy=0x7fe750073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.097+0000 7fe74f59e640 1 -- 172.21.15.50:0/3052842638 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe750072850 con 0x7fe7500b0690 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.098+0000 7fe74f59e640 1 --2- 172.21.15.50:0/3052842638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7500addf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe748009e50 tx=0x7fe74802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d104720bb1cdf31 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.098+0000 7fe74e59c640 1 -- 172.21.15.50:0/3052842638 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74802ca70 con 0x7fe7500b0690 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.098+0000 7fe74e59c640 1 -- 172.21.15.50:0/3052842638 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74802cc10 con 0x7fe7500b0690 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.098+0000 7fe74e59c640 1 -- 172.21.15.50:0/3052842638 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480343f0 con 0x7fe7500b0690 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 -- 172.21.15.50:0/3052842638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 msgr2=0x7fe7500addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3052842638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7500addf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe748009e50 tx=0x7fe74802bed0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 -- 172.21.15.50:0/3052842638 shutdown_connections 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3052842638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7500addf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3052842638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75006bcf0 0x7fe7500ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 -- 172.21.15.50:0/3052842638 >> 172.21.15.50:0/3052842638 conn(0x7fe7500b6540 msgr2=0x7fe7500b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 -- 172.21.15.50:0/3052842638 shutdown_connections 2024-04-02T16:22:29.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.099+0000 7fe7551e4640 1 -- 172.21.15.50:0/3052842638 wait complete. 2024-04-02T16:22:29.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.100+0000 7fe7551e4640 1 Processor -- start 2024-04-02T16:22:29.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.100+0000 7fe7551e4640 1 -- start start 2024-04-02T16:22:29.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75006bcf0 0x7fe750153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7501569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe750161b80 0x7fe750156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe750073760 con 0x7fe7500b0690 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe7500b01e0 con 0x7fe75006bcf0 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe7500738e0 con 0x7fe750161b80 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7501569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7501569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45364/0 (socket says 172.21.15.50:45364) 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 -- 172.21.15.50:0/3833401442 learned_addr learned my addr 172.21.15.50:0/3833401442 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe74f59e640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe750161b80 0x7fe750156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe74ed9d640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75006bcf0 0x7fe750153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 -- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe750161b80 msgr2=0x7fe750156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe750161b80 0x7fe750156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 -- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75006bcf0 msgr2=0x7fe750153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75006bcf0 0x7fe750153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 -- 172.21.15.50:0/3833401442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7500d0700 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe74f59e640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe750161b80 0x7fe750156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe747fff640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7501569b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe73800d830 tx=0x7fe73800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7477fe640 1 -- 172.21.15.50:0/3833401442 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73800b6c0 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7477fe640 1 -- 172.21.15.50:0/3833401442 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe738019070 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7501552a0 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7477fe640 1 -- 172.21.15.50:0/3833401442 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73800bac0 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.101+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe750154830 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.103+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7500756f0 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.103+0000 7fe7477fe640 1 -- 172.21.15.50:0/3833401442 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe73800bc60 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.103+0000 7fe7477fe640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe73006c920 0x7fe73006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.103+0000 7fe7477fe640 1 -- 172.21.15.50:0/3833401442 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe73808c370 con 0x7fe7500b0690 2024-04-02T16:22:29.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.104+0000 7fe74ed9d640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe73006c920 0x7fe73006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.104+0000 7fe74ed9d640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe73006c920 0x7fe73006ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe740007fd0 tx=0x7fe740009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.108+0000 7fe7477fe640 1 -- 172.21.15.50:0/3833401442 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe73805a450 con 0x7fe7500b0690 2024-04-02T16:22:29.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.288+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 --> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe750154300 con 0x7fe73006c920 2024-04-02T16:22:29.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.296+0000 7fe7477fe640 1 -- 172.21.15.50:0/3833401442 <== mgr.4112 v2:172.21.15.188:6832/2785600878 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7fe750154300 con 0x7fe73006c920 2024-04-02T16:22:29.301 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:29.303 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-04-02T16:22:29.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe73006c920 msgr2=0x7fe73006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe73006c920 0x7fe73006ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe740007fd0 tx=0x7fe740009040 comp rx=0 tx=0).stop 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 msgr2=0x7fe7501569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7501569b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe73800d830 tx=0x7fe73800dcf0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 shutdown_connections 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe750161b80 0x7fe750156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe73006c920 0x7fe73006ede0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7500b0690 0x7fe7501569b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 --2- 172.21.15.50:0/3833401442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75006bcf0 0x7fe750153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 >> 172.21.15.50:0/3833401442 conn(0x7fe7500b6540 msgr2=0x7fe7500b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.299+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 shutdown_connections 2024-04-02T16:22:29.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.300+0000 7fe7551e4640 1 -- 172.21.15.50:0/3833401442 wait complete. 2024-04-02T16:22:29.312 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-02T16:22:28.672801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236104,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738616,"statfs":{"total":773094113280,"available":772852342784,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003105"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434993+0000","last_change":"2024-04-02T16:22:00.392396+0000","last_active":"2024-04-02T16:22:07.434993+0000","last_peered":"2024-04-02T16:22:07.434993+0000","last_clean":"2024-04-02T16:22:07.434993+0000","last_became_active":"2024-04-02T16:22:00.392089+0000","last_became_peered":"2024-04-02T16:22:00.392089+0000","last_unstale":"2024-04-02T16:22:07.434993+0000","last_undegraded":"2024-04-02T16:22:07.434993+0000","last_fullsized":"2024-04-02T16:22:07.434993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428539+0000","last_change":"2024-04-02T16:22:01.410877+0000","last_active":"2024-04-02T16:22:07.428539+0000","last_peered":"2024-04-02T16:22:07.428539+0000","last_clean":"2024-04-02T16:22:07.428539+0000","last_became_active":"2024-04-02T16:22:01.410574+0000","last_became_peered":"2024-04-02T16:22:01.410574+0000","last_unstale":"2024-04-02T16:22:07.428539+0000","last_undegraded":"2024-04-02T16:22:07.428539+0000","last_fullsized":"2024-04-02T16:22:07.428539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427767+0000","last_change":"2024-04-02T16:22:05.817506+0000","last_active":"2024-04-02T16:22:07.427767+0000","last_peered":"2024-04-02T16:22:07.427767+0000","last_clean":"2024-04-02T16:22:07.427767+0000","last_became_active":"2024-04-02T16:22:05.817145+0000","last_became_peered":"2024-04-02T16:22:05.817145+0000","last_unstale":"2024-04-02T16:22:07.427767+0000","last_undegraded":"2024-04-02T16:22:07.427767+0000","last_fullsized":"2024-04-02T16:22:07.427767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:30.473603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428664+0000","last_change":"2024-04-02T16:22:00.391720+0000","last_active":"2024-04-02T16:22:07.428664+0000","last_peered":"2024-04-02T16:22:07.428664+0000","last_clean":"2024-04-02T16:22:07.428664+0000","last_became_active":"2024-04-02T16:22:00.391262+0000","last_became_peered":"2024-04-02T16:22:00.391262+0000","last_unstale":"2024-04-02T16:22:07.428664+0000","last_undegraded":"2024-04-02T16:22:07.428664+0000","last_fullsized":"2024-04-02T16:22:07.428664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434717+0000","last_change":"2024-04-02T16:22:01.403096+0000","last_active":"2024-04-02T16:22:07.434717+0000","last_peered":"2024-04-02T16:22:07.434717+0000","last_clean":"2024-04-02T16:22:07.434717+0000","last_became_active":"2024-04-02T16:22:01.402805+0000","last_became_peered":"2024-04-02T16:22:01.402805+0000","last_unstale":"2024-04-02T16:22:07.434717+0000","last_undegraded":"2024-04-02T16:22:07.434717+0000","last_fullsized":"2024-04-02T16:22:07.434717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428967+0000","last_change":"2024-04-02T16:22:05.447225+0000","last_active":"2024-04-02T16:22:07.428967+0000","last_peered":"2024-04-02T16:22:07.428967+0000","last_clean":"2024-04-02T16:22:07.428967+0000","last_became_active":"2024-04-02T16:22:05.446843+0000","last_became_peered":"2024-04-02T16:22:05.446843+0000","last_unstale":"2024-04-02T16:22:07.428967+0000","last_undegraded":"2024-04-02T16:22:07.428967+0000","last_fullsized":"2024-04-02T16:22:07.428967+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:00.390875+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:00.390501+0000","last_became_peered":"2024-04-02T16:22:00.390501+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430302+0000","last_change":"2024-04-02T16:22:01.410933+0000","last_active":"2024-04-02T16:22:07.430302+0000","last_peered":"2024-04-02T16:22:07.430302+0000","last_clean":"2024-04-02T16:22:07.430302+0000","last_became_active":"2024-04-02T16:22:01.410242+0000","last_became_peered":"2024-04-02T16:22:01.410242+0000","last_unstale":"2024-04-02T16:22:07.430302+0000","last_undegraded":"2024-04-02T16:22:07.430302+0000","last_fullsized":"2024-04-02T16:22:07.430302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429559+0000","last_change":"2024-04-02T16:22:05.440230+0000","last_active":"2024-04-02T16:22:07.429559+0000","last_peered":"2024-04-02T16:22:07.429559+0000","last_clean":"2024-04-02T16:22:07.429559+0000","last_became_active":"2024-04-02T16:22:05.439599+0000","last_became_peered":"2024-04-02T16:22:05.439599+0000","last_unstale":"2024-04-02T16:22:07.429559+0000","last_undegraded":"2024-04-02T16:22:07.429559+0000","last_fullsized":"2024-04-02T16:22:07.429559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:35.350953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434504+0000","last_change":"2024-04-02T16:22:00.392521+0000","last_active":"2024-04-02T16:22:07.434504+0000","last_peered":"2024-04-02T16:22:07.434504+0000","last_clean":"2024-04-02T16:22:07.434504+0000","last_became_active":"2024-04-02T16:22:00.392244+0000","last_became_peered":"2024-04-02T16:22:00.392244+0000","last_unstale":"2024-04-02T16:22:07.434504+0000","last_undegraded":"2024-04-02T16:22:07.434504+0000","last_fullsized":"2024-04-02T16:22:07.434504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430629+0000","last_change":"2024-04-02T16:22:01.406612+0000","last_active":"2024-04-02T16:22:07.430629+0000","last_peered":"2024-04-02T16:22:07.430629+0000","last_clean":"2024-04-02T16:22:07.430629+0000","last_became_active":"2024-04-02T16:22:01.404789+0000","last_became_peered":"2024-04-02T16:22:01.404789+0000","last_unstale":"2024-04-02T16:22:07.430629+0000","last_undegraded":"2024-04-02T16:22:07.430629+0000","last_fullsized":"2024-04-02T16:22:07.430629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429046+0000","last_change":"2024-04-02T16:22:05.445479+0000","last_active":"2024-04-02T16:22:07.429046+0000","last_peered":"2024-04-02T16:22:07.429046+0000","last_clean":"2024-04-02T16:22:07.429046+0000","last_became_active":"2024-04-02T16:22:05.445022+0000","last_became_peered":"2024-04-02T16:22:05.445022+0000","last_unstale":"2024-04-02T16:22:07.429046+0000","last_undegraded":"2024-04-02T16:22:07.429046+0000","last_fullsized":"2024-04-02T16:22:07.429046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.404940+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.404670+0000","last_became_peered":"2024-04-02T16:22:01.404670+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434832+0000","last_change":"2024-04-02T16:22:00.390811+0000","last_active":"2024-04-02T16:22:07.434832+0000","last_peered":"2024-04-02T16:22:07.434832+0000","last_clean":"2024-04-02T16:22:07.434832+0000","last_became_active":"2024-04-02T16:22:00.390515+0000","last_became_peered":"2024-04-02T16:22:00.390515+0000","last_unstale":"2024-04-02T16:22:07.434832+0000","last_undegraded":"2024-04-02T16:22:07.434832+0000","last_fullsized":"2024-04-02T16:22:07.434832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429947+0000","last_change":"2024-04-02T16:22:05.445431+0000","last_active":"2024-04-02T16:22:07.429947+0000","last_peered":"2024-04-02T16:22:07.429947+0000","last_clean":"2024-04-02T16:22:07.429947+0000","last_became_active":"2024-04-02T16:22:05.445014+0000","last_became_peered":"2024-04-02T16:22:05.445014+0000","last_unstale":"2024-04-02T16:22:07.429947+0000","last_undegraded":"2024-04-02T16:22:07.429947+0000","last_fullsized":"2024-04-02T16:22:07.429947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428171+0000","last_change":"2024-04-02T16:22:01.409289+0000","last_active":"2024-04-02T16:22:07.428171+0000","last_peered":"2024-04-02T16:22:07.428171+0000","last_clean":"2024-04-02T16:22:07.428171+0000","last_became_active":"2024-04-02T16:22:01.408934+0000","last_became_peered":"2024-04-02T16:22:01.408934+0000","last_unstale":"2024-04-02T16:22:07.428171+0000","last_undegraded":"2024-04-02T16:22:07.428171+0000","last_fullsized":"2024-04-02T16:22:07.428171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434963+0000","last_change":"2024-04-02T16:22:00.390727+0000","last_active":"2024-04-02T16:22:07.434963+0000","last_peered":"2024-04-02T16:22:07.434963+0000","last_clean":"2024-04-02T16:22:07.434963+0000","last_became_active":"2024-04-02T16:22:00.390131+0000","last_became_peered":"2024-04-02T16:22:00.390131+0000","last_unstale":"2024-04-02T16:22:07.434963+0000","last_undegraded":"2024-04-02T16:22:07.434963+0000","last_fullsized":"2024-04-02T16:22:07.434963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429330+0000","last_change":"2024-04-02T16:22:05.447195+0000","last_active":"2024-04-02T16:22:07.429330+0000","last_peered":"2024-04-02T16:22:07.429330+0000","last_clean":"2024-04-02T16:22:07.429330+0000","last_became_active":"2024-04-02T16:22:05.446753+0000","last_became_peered":"2024-04-02T16:22:05.446753+0000","last_unstale":"2024-04-02T16:22:07.429330+0000","last_undegraded":"2024-04-02T16:22:07.429330+0000","last_fullsized":"2024-04-02T16:22:07.429330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427951+0000","last_change":"2024-04-02T16:22:01.400634+0000","last_active":"2024-04-02T16:22:07.427951+0000","last_peered":"2024-04-02T16:22:07.427951+0000","last_clean":"2024-04-02T16:22:07.427951+0000","last_became_active":"2024-04-02T16:22:01.400309+0000","last_became_peered":"2024-04-02T16:22:01.400309+0000","last_unstale":"2024-04-02T16:22:07.427951+0000","last_undegraded":"2024-04-02T16:22:07.427951+0000","last_fullsized":"2024-04-02T16:22:07.427951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434579+0000","last_change":"2024-04-02T16:22:00.391494+0000","last_active":"2024-04-02T16:22:07.434579+0000","last_peered":"2024-04-02T16:22:07.434579+0000","last_clean":"2024-04-02T16:22:07.434579+0000","last_became_active":"2024-04-02T16:22:00.391190+0000","last_became_peered":"2024-04-02T16:22:00.391190+0000","last_unstale":"2024-04-02T16:22:07.434579+0000","last_undegraded":"2024-04-02T16:22:07.434579+0000","last_fullsized":"2024-04-02T16:22:07.434579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435264+0000","last_change":"2024-04-02T16:22:05.811534+0000","last_active":"2024-04-02T16:22:07.435264+0000","last_peered":"2024-04-02T16:22:07.435264+0000","last_clean":"2024-04-02T16:22:07.435264+0000","last_became_active":"2024-04-02T16:22:05.811134+0000","last_became_peered":"2024-04-02T16:22:05.811134+0000","last_unstale":"2024-04-02T16:22:07.435264+0000","last_undegraded":"2024-04-02T16:22:07.435264+0000","last_fullsized":"2024-04-02T16:22:07.435264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430330+0000","last_change":"2024-04-02T16:22:01.399287+0000","last_active":"2024-04-02T16:22:07.430330+0000","last_peered":"2024-04-02T16:22:07.430330+0000","last_clean":"2024-04-02T16:22:07.430330+0000","last_became_active":"2024-04-02T16:22:01.398956+0000","last_became_peered":"2024-04-02T16:22:01.398956+0000","last_unstale":"2024-04-02T16:22:07.430330+0000","last_undegraded":"2024-04-02T16:22:07.430330+0000","last_fullsized":"2024-04-02T16:22:07.430330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:50.927884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434912+0000","last_change":"2024-04-02T16:22:00.391289+0000","last_active":"2024-04-02T16:22:07.434912+0000","last_peered":"2024-04-02T16:22:07.434912+0000","last_clean":"2024-04-02T16:22:07.434912+0000","last_became_active":"2024-04-02T16:22:00.390292+0000","last_became_peered":"2024-04-02T16:22:00.390292+0000","last_unstale":"2024-04-02T16:22:07.434912+0000","last_undegraded":"2024-04-02T16:22:07.434912+0000","last_fullsized":"2024-04-02T16:22:07.434912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429938+0000","last_change":"2024-04-02T16:22:05.440165+0000","last_active":"2024-04-02T16:22:07.429938+0000","last_peered":"2024-04-02T16:22:07.429938+0000","last_clean":"2024-04-02T16:22:07.429938+0000","last_became_active":"2024-04-02T16:22:05.439732+0000","last_became_peered":"2024-04-02T16:22:05.439732+0000","last_unstale":"2024-04-02T16:22:07.429938+0000","last_undegraded":"2024-04-02T16:22:07.429938+0000","last_fullsized":"2024-04-02T16:22:07.429938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434684+0000","last_change":"2024-04-02T16:22:01.404915+0000","last_active":"2024-04-02T16:22:07.434684+0000","last_peered":"2024-04-02T16:22:07.434684+0000","last_clean":"2024-04-02T16:22:07.434684+0000","last_became_active":"2024-04-02T16:22:01.402769+0000","last_became_peered":"2024-04-02T16:22:01.402769+0000","last_unstale":"2024-04-02T16:22:07.434684+0000","last_undegraded":"2024-04-02T16:22:07.434684+0000","last_fullsized":"2024-04-02T16:22:07.434684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:45.810039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435410+0000","last_change":"2024-04-02T16:22:00.393479+0000","last_active":"2024-04-02T16:22:07.435410+0000","last_peered":"2024-04-02T16:22:07.435410+0000","last_clean":"2024-04-02T16:22:07.435410+0000","last_became_active":"2024-04-02T16:22:00.393178+0000","last_became_peered":"2024-04-02T16:22:00.393178+0000","last_unstale":"2024-04-02T16:22:07.435410+0000","last_undegraded":"2024-04-02T16:22:07.435410+0000","last_fullsized":"2024-04-02T16:22:07.435410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430212+0000","last_change":"2024-04-02T16:22:05.815460+0000","last_active":"2024-04-02T16:22:07.430212+0000","last_peered":"2024-04-02T16:22:07.430212+0000","last_clean":"2024-04-02T16:22:07.430212+0000","last_became_active":"2024-04-02T16:22:05.814988+0000","last_became_peered":"2024-04-02T16:22:05.814988+0000","last_unstale":"2024-04-02T16:22:07.430212+0000","last_undegraded":"2024-04-02T16:22:07.430212+0000","last_fullsized":"2024-04-02T16:22:07.430212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:01.404044+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:01.403758+0000","last_became_peered":"2024-04-02T16:22:01.403758+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435274+0000","last_change":"2024-04-02T16:22:00.394933+0000","last_active":"2024-04-02T16:22:07.435274+0000","last_peered":"2024-04-02T16:22:07.435274+0000","last_clean":"2024-04-02T16:22:07.435274+0000","last_became_active":"2024-04-02T16:22:00.394672+0000","last_became_peered":"2024-04-02T16:22:00.394672+0000","last_unstale":"2024-04-02T16:22:07.435274+0000","last_undegraded":"2024-04-02T16:22:07.435274+0000","last_fullsized":"2024-04-02T16:22:07.435274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:44.086272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427719+0000","last_change":"2024-04-02T16:22:05.816404+0000","last_active":"2024-04-02T16:22:07.427719+0000","last_peered":"2024-04-02T16:22:07.427719+0000","last_clean":"2024-04-02T16:22:07.427719+0000","last_became_active":"2024-04-02T16:22:05.816011+0000","last_became_peered":"2024-04-02T16:22:05.816011+0000","last_unstale":"2024-04-02T16:22:07.427719+0000","last_undegraded":"2024-04-02T16:22:07.427719+0000","last_fullsized":"2024-04-02T16:22:07.427719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:58.796179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434982+0000","last_change":"2024-04-02T16:22:01.405660+0000","last_active":"2024-04-02T16:22:07.434982+0000","last_peered":"2024-04-02T16:22:07.434982+0000","last_clean":"2024-04-02T16:22:07.434982+0000","last_became_active":"2024-04-02T16:22:01.405343+0000","last_became_peered":"2024-04-02T16:22:01.405343+0000","last_unstale":"2024-04-02T16:22:07.434982+0000","last_undegraded":"2024-04-02T16:22:07.434982+0000","last_fullsized":"2024-04-02T16:22:07.434982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435169+0000","last_change":"2024-04-02T16:22:00.391246+0000","last_active":"2024-04-02T16:22:07.435169+0000","last_peered":"2024-04-02T16:22:07.435169+0000","last_clean":"2024-04-02T16:22:07.435169+0000","last_became_active":"2024-04-02T16:22:00.390836+0000","last_became_peered":"2024-04-02T16:22:00.390836+0000","last_unstale":"2024-04-02T16:22:07.435169+0000","last_undegraded":"2024-04-02T16:22:07.435169+0000","last_fullsized":"2024-04-02T16:22:07.435169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:22.952130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435359+0000","last_change":"2024-04-02T16:22:05.815150+0000","last_active":"2024-04-02T16:22:07.435359+0000","last_peered":"2024-04-02T16:22:07.435359+0000","last_clean":"2024-04-02T16:22:07.435359+0000","last_became_active":"2024-04-02T16:22:05.814788+0000","last_became_peered":"2024-04-02T16:22:05.814788+0000","last_unstale":"2024-04-02T16:22:07.435359+0000","last_undegraded":"2024-04-02T16:22:07.435359+0000","last_fullsized":"2024-04-02T16:22:07.435359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430743+0000","last_change":"2024-04-02T16:22:01.404482+0000","last_active":"2024-04-02T16:22:07.430743+0000","last_peered":"2024-04-02T16:22:07.430743+0000","last_clean":"2024-04-02T16:22:07.430743+0000","last_became_active":"2024-04-02T16:22:01.402612+0000","last_became_peered":"2024-04-02T16:22:01.402612+0000","last_unstale":"2024-04-02T16:22:07.430743+0000","last_undegraded":"2024-04-02T16:22:07.430743+0000","last_fullsized":"2024-04-02T16:22:07.430743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435056+0000","last_change":"2024-04-02T16:22:00.388850+0000","last_active":"2024-04-02T16:22:07.435056+0000","last_peered":"2024-04-02T16:22:07.435056+0000","last_clean":"2024-04-02T16:22:07.435056+0000","last_became_active":"2024-04-02T16:22:00.388569+0000","last_became_peered":"2024-04-02T16:22:00.388569+0000","last_unstale":"2024-04-02T16:22:07.435056+0000","last_undegraded":"2024-04-02T16:22:07.435056+0000","last_fullsized":"2024-04-02T16:22:07.435056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434575+0000","last_change":"2024-04-02T16:22:05.445952+0000","last_active":"2024-04-02T16:22:07.434575+0000","last_peered":"2024-04-02T16:22:07.434575+0000","last_clean":"2024-04-02T16:22:07.434575+0000","last_became_active":"2024-04-02T16:22:05.445442+0000","last_became_peered":"2024-04-02T16:22:05.445442+0000","last_unstale":"2024-04-02T16:22:07.434575+0000","last_undegraded":"2024-04-02T16:22:07.434575+0000","last_fullsized":"2024-04-02T16:22:07.434575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429867+0000","last_change":"2024-04-02T16:22:01.405033+0000","last_active":"2024-04-02T16:22:07.429867+0000","last_peered":"2024-04-02T16:22:07.429867+0000","last_clean":"2024-04-02T16:22:07.429867+0000","last_became_active":"2024-04-02T16:22:01.404647+0000","last_became_peered":"2024-04-02T16:22:01.404647+0000","last_unstale":"2024-04-02T16:22:07.429867+0000","last_undegraded":"2024-04-02T16:22:07.429867+0000","last_fullsized":"2024-04-02T16:22:07.429867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:03.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429937+0000","last_change":"2024-04-02T16:22:00.390557+0000","last_active":"2024-04-02T16:22:07.429937+0000","last_peered":"2024-04-02T16:22:07.429937+0000","last_clean":"2024-04-02T16:22:07.429937+0000","last_became_active":"2024-04-02T16:22:00.390250+0000","last_became_peered":"2024-04-02T16:22:00.390250+0000","last_unstale":"2024-04-02T16:22:07.429937+0000","last_undegraded":"2024-04-02T16:22:07.429937+0000","last_fullsized":"2024-04-02T16:22:07.429937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434997+0000","last_change":"2024-04-02T16:22:05.448031+0000","last_active":"2024-04-02T16:22:07.434997+0000","last_peered":"2024-04-02T16:22:07.434997+0000","last_clean":"2024-04-02T16:22:07.434997+0000","last_became_active":"2024-04-02T16:22:05.447648+0000","last_became_peered":"2024-04-02T16:22:05.447648+0000","last_unstale":"2024-04-02T16:22:07.434997+0000","last_undegraded":"2024-04-02T16:22:07.434997+0000","last_fullsized":"2024-04-02T16:22:07.434997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433811+0000","last_change":"2024-04-02T16:22:01.406024+0000","last_active":"2024-04-02T16:22:07.433811+0000","last_peered":"2024-04-02T16:22:07.433811+0000","last_clean":"2024-04-02T16:22:07.433811+0000","last_became_active":"2024-04-02T16:22:01.405628+0000","last_became_peered":"2024-04-02T16:22:01.405628+0000","last_unstale":"2024-04-02T16:22:07.433811+0000","last_undegraded":"2024-04-02T16:22:07.433811+0000","last_fullsized":"2024-04-02T16:22:07.433811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433971+0000","last_change":"2024-04-02T16:22:00.384113+0000","last_active":"2024-04-02T16:22:07.433971+0000","last_peered":"2024-04-02T16:22:07.433971+0000","last_clean":"2024-04-02T16:22:07.433971+0000","last_became_active":"2024-04-02T16:22:00.383827+0000","last_became_peered":"2024-04-02T16:22:00.383827+0000","last_unstale":"2024-04-02T16:22:07.433971+0000","last_undegraded":"2024-04-02T16:22:07.433971+0000","last_fullsized":"2024-04-02T16:22:07.433971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435258+0000","last_change":"2024-04-02T16:22:05.448007+0000","last_active":"2024-04-02T16:22:07.435258+0000","last_peered":"2024-04-02T16:22:07.435258+0000","last_clean":"2024-04-02T16:22:07.435258+0000","last_became_active":"2024-04-02T16:22:05.447605+0000","last_became_peered":"2024-04-02T16:22:05.447605+0000","last_unstale":"2024-04-02T16:22:07.435258+0000","last_undegraded":"2024-04-02T16:22:07.435258+0000","last_fullsized":"2024-04-02T16:22:07.435258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434278+0000","last_change":"2024-04-02T16:22:01.394732+0000","last_active":"2024-04-02T16:22:07.434278+0000","last_peered":"2024-04-02T16:22:07.434278+0000","last_clean":"2024-04-02T16:22:07.434278+0000","last_became_active":"2024-04-02T16:22:01.394437+0000","last_became_peered":"2024-04-02T16:22:01.394437+0000","last_unstale":"2024-04-02T16:22:07.434278+0000","last_undegraded":"2024-04-02T16:22:07.434278+0000","last_fullsized":"2024-04-02T16:22:07.434278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:26:16.963906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434233+0000","last_change":"2024-04-02T16:22:00.391466+0000","last_active":"2024-04-02T16:22:07.434233+0000","last_peered":"2024-04-02T16:22:07.434233+0000","last_clean":"2024-04-02T16:22:07.434233+0000","last_became_active":"2024-04-02T16:22:00.391148+0000","last_became_peered":"2024-04-02T16:22:00.391148+0000","last_unstale":"2024-04-02T16:22:07.434233+0000","last_undegraded":"2024-04-02T16:22:07.434233+0000","last_fullsized":"2024-04-02T16:22:07.434233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:05:02.245621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434509+0000","last_change":"2024-04-02T16:22:05.811459+0000","last_active":"2024-04-02T16:22:07.434509+0000","last_peered":"2024-04-02T16:22:07.434509+0000","last_clean":"2024-04-02T16:22:07.434509+0000","last_became_active":"2024-04-02T16:22:05.811036+0000","last_became_peered":"2024-04-02T16:22:05.811036+0000","last_unstale":"2024-04-02T16:22:07.434509+0000","last_undegraded":"2024-04-02T16:22:07.434509+0000","last_fullsized":"2024-04-02T16:22:07.434509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428029+0000","last_change":"2024-04-02T16:22:01.410992+0000","last_active":"2024-04-02T16:22:07.428029+0000","last_peered":"2024-04-02T16:22:07.428029+0000","last_clean":"2024-04-02T16:22:07.428029+0000","last_became_active":"2024-04-02T16:22:01.410701+0000","last_became_peered":"2024-04-02T16:22:01.410701+0000","last_unstale":"2024-04-02T16:22:07.428029+0000","last_undegraded":"2024-04-02T16:22:07.428029+0000","last_fullsized":"2024-04-02T16:22:07.428029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429896+0000","last_change":"2024-04-02T16:22:00.382563+0000","last_active":"2024-04-02T16:22:07.429896+0000","last_peered":"2024-04-02T16:22:07.429896+0000","last_clean":"2024-04-02T16:22:07.429896+0000","last_became_active":"2024-04-02T16:22:00.382285+0000","last_became_peered":"2024-04-02T16:22:00.382285+0000","last_unstale":"2024-04-02T16:22:07.429896+0000","last_undegraded":"2024-04-02T16:22:07.429896+0000","last_fullsized":"2024-04-02T16:22:07.429896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:21:49.851609+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:21:49.850564+0000","last_became_peered":"2024-04-02T16:21:49.850564+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_clean_scrub_stamp":"2024-04-02T16:21:48.692321+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.825754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434765+0000","last_change":"2024-04-02T16:22:05.816624+0000","last_active":"2024-04-02T16:22:07.434765+0000","last_peered":"2024-04-02T16:22:07.434765+0000","last_clean":"2024-04-02T16:22:07.434765+0000","last_became_active":"2024-04-02T16:22:05.816230+0000","last_became_peered":"2024-04-02T16:22:05.816230+0000","last_unstale":"2024-04-02T16:22:07.434765+0000","last_undegraded":"2024-04-02T16:22:07.434765+0000","last_fullsized":"2024-04-02T16:22:07.434765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:49.037743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434871+0000","last_change":"2024-04-02T16:22:01.412438+0000","last_active":"2024-04-02T16:22:07.434871+0000","last_peered":"2024-04-02T16:22:07.434871+0000","last_clean":"2024-04-02T16:22:07.434871+0000","last_became_active":"2024-04-02T16:22:01.411845+0000","last_became_peered":"2024-04-02T16:22:01.411845+0000","last_unstale":"2024-04-02T16:22:07.434871+0000","last_undegraded":"2024-04-02T16:22:07.434871+0000","last_fullsized":"2024-04-02T16:22:07.434871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:08.681756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434349+0000","last_change":"2024-04-02T16:22:00.381230+0000","last_active":"2024-04-02T16:22:07.434349+0000","last_peered":"2024-04-02T16:22:07.434349+0000","last_clean":"2024-04-02T16:22:07.434349+0000","last_became_active":"2024-04-02T16:22:00.380907+0000","last_became_peered":"2024-04-02T16:22:00.380907+0000","last_unstale":"2024-04-02T16:22:07.434349+0000","last_undegraded":"2024-04-02T16:22:07.434349+0000","last_fullsized":"2024-04-02T16:22:07.434349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434320+0000","last_change":"2024-04-02T16:22:05.815548+0000","last_active":"2024-04-02T16:22:07.434320+0000","last_peered":"2024-04-02T16:22:07.434320+0000","last_clean":"2024-04-02T16:22:07.434320+0000","last_became_active":"2024-04-02T16:22:05.815073+0000","last_became_peered":"2024-04-02T16:22:05.815073+0000","last_unstale":"2024-04-02T16:22:07.434320+0000","last_undegraded":"2024-04-02T16:22:07.434320+0000","last_fullsized":"2024-04-02T16:22:07.434320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:12.728804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430426+0000","last_change":"2024-04-02T16:22:01.407537+0000","last_active":"2024-04-02T16:22:07.430426+0000","last_peered":"2024-04-02T16:22:07.430426+0000","last_clean":"2024-04-02T16:22:07.430426+0000","last_became_active":"2024-04-02T16:22:01.403892+0000","last_became_peered":"2024-04-02T16:22:01.403892+0000","last_unstale":"2024-04-02T16:22:07.430426+0000","last_undegraded":"2024-04-02T16:22:07.430426+0000","last_fullsized":"2024-04-02T16:22:07.430426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434911+0000","last_change":"2024-04-02T16:22:00.387875+0000","last_active":"2024-04-02T16:22:07.434911+0000","last_peered":"2024-04-02T16:22:07.434911+0000","last_clean":"2024-04-02T16:22:07.434911+0000","last_became_active":"2024-04-02T16:22:00.387562+0000","last_became_peered":"2024-04-02T16:22:00.387562+0000","last_unstale":"2024-04-02T16:22:07.434911+0000","last_undegraded":"2024-04-02T16:22:07.434911+0000","last_fullsized":"2024-04-02T16:22:07.434911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435508+0000","last_change":"2024-04-02T16:22:05.450999+0000","last_active":"2024-04-02T16:22:07.435508+0000","last_peered":"2024-04-02T16:22:07.435508+0000","last_clean":"2024-04-02T16:22:07.435508+0000","last_became_active":"2024-04-02T16:22:05.450407+0000","last_became_peered":"2024-04-02T16:22:05.450407+0000","last_unstale":"2024-04-02T16:22:07.435508+0000","last_undegraded":"2024-04-02T16:22:07.435508+0000","last_fullsized":"2024-04-02T16:22:07.435508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435483+0000","last_change":"2024-04-02T16:22:01.410592+0000","last_active":"2024-04-02T16:22:07.435483+0000","last_peered":"2024-04-02T16:22:07.435483+0000","last_clean":"2024-04-02T16:22:07.435483+0000","last_became_active":"2024-04-02T16:22:01.410288+0000","last_became_peered":"2024-04-02T16:22:01.410288+0000","last_unstale":"2024-04-02T16:22:07.435483+0000","last_undegraded":"2024-04-02T16:22:07.435483+0000","last_fullsized":"2024-04-02T16:22:07.435483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:58.411440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434019+0000","last_change":"2024-04-02T16:22:00.389059+0000","last_active":"2024-04-02T16:22:07.434019+0000","last_peered":"2024-04-02T16:22:07.434019+0000","last_clean":"2024-04-02T16:22:07.434019+0000","last_became_active":"2024-04-02T16:22:00.388780+0000","last_became_peered":"2024-04-02T16:22:00.388780+0000","last_unstale":"2024-04-02T16:22:07.434019+0000","last_undegraded":"2024-04-02T16:22:07.434019+0000","last_fullsized":"2024-04-02T16:22:07.434019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430338+0000","last_change":"2024-04-02T16:22:05.811513+0000","last_active":"2024-04-02T16:22:07.430338+0000","last_peered":"2024-04-02T16:22:07.430338+0000","last_clean":"2024-04-02T16:22:07.430338+0000","last_became_active":"2024-04-02T16:22:05.811011+0000","last_became_peered":"2024-04-02T16:22:05.811011+0000","last_unstale":"2024-04-02T16:22:07.430338+0000","last_undegraded":"2024-04-02T16:22:07.430338+0000","last_fullsized":"2024-04-02T16:22:07.430338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:05:03.410325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435147+0000","last_change":"2024-04-02T16:22:01.404383+0000","last_active":"2024-04-02T16:22:07.435147+0000","last_peered":"2024-04-02T16:22:07.435147+0000","last_clean":"2024-04-02T16:22:07.435147+0000","last_became_active":"2024-04-02T16:22:01.404105+0000","last_became_peered":"2024-04-02T16:22:01.404105+0000","last_unstale":"2024-04-02T16:22:07.435147+0000","last_undegraded":"2024-04-02T16:22:07.435147+0000","last_fullsized":"2024-04-02T16:22:07.435147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429344+0000","last_change":"2024-04-02T16:22:00.385712+0000","last_active":"2024-04-02T16:22:07.429344+0000","last_peered":"2024-04-02T16:22:07.429344+0000","last_clean":"2024-04-02T16:22:07.429344+0000","last_became_active":"2024-04-02T16:22:00.385374+0000","last_became_peered":"2024-04-02T16:22:00.385374+0000","last_unstale":"2024-04-02T16:22:07.429344+0000","last_undegraded":"2024-04-02T16:22:07.429344+0000","last_fullsized":"2024-04-02T16:22:07.429344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430015+0000","last_change":"2024-04-02T16:22:05.439393+0000","last_active":"2024-04-02T16:22:07.430015+0000","last_peered":"2024-04-02T16:22:07.430015+0000","last_clean":"2024-04-02T16:22:07.430015+0000","last_became_active":"2024-04-02T16:22:05.439022+0000","last_became_peered":"2024-04-02T16:22:05.439022+0000","last_unstale":"2024-04-02T16:22:07.430015+0000","last_undegraded":"2024-04-02T16:22:07.430015+0000","last_fullsized":"2024-04-02T16:22:07.430015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:12:06.622201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434659+0000","last_change":"2024-04-02T16:22:01.412452+0000","last_active":"2024-04-02T16:22:07.434659+0000","last_peered":"2024-04-02T16:22:07.434659+0000","last_clean":"2024-04-02T16:22:07.434659+0000","last_became_active":"2024-04-02T16:22:01.412160+0000","last_became_peered":"2024-04-02T16:22:01.412160+0000","last_unstale":"2024-04-02T16:22:07.434659+0000","last_undegraded":"2024-04-02T16:22:07.434659+0000","last_fullsized":"2024-04-02T16:22:07.434659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433532+0000","last_change":"2024-04-02T16:22:00.386167+0000","last_active":"2024-04-02T16:22:07.433532+0000","last_peered":"2024-04-02T16:22:07.433532+0000","last_clean":"2024-04-02T16:22:07.433532+0000","last_became_active":"2024-04-02T16:22:00.385887+0000","last_became_peered":"2024-04-02T16:22:00.385887+0000","last_unstale":"2024-04-02T16:22:07.433532+0000","last_undegraded":"2024-04-02T16:22:07.433532+0000","last_fullsized":"2024-04-02T16:22:07.433532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434360+0000","last_change":"2024-04-02T16:22:05.451144+0000","last_active":"2024-04-02T16:22:07.434360+0000","last_peered":"2024-04-02T16:22:07.434360+0000","last_clean":"2024-04-02T16:22:07.434360+0000","last_became_active":"2024-04-02T16:22:05.450690+0000","last_became_peered":"2024-04-02T16:22:05.450690+0000","last_unstale":"2024-04-02T16:22:07.434360+0000","last_undegraded":"2024-04-02T16:22:07.434360+0000","last_fullsized":"2024-04-02T16:22:07.434360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:02.458487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434308+0000","last_change":"2024-04-02T16:22:01.409326+0000","last_active":"2024-04-02T16:22:07.434308+0000","last_peered":"2024-04-02T16:22:07.434308+0000","last_clean":"2024-04-02T16:22:07.434308+0000","last_became_active":"2024-04-02T16:22:01.408710+0000","last_became_peered":"2024-04-02T16:22:01.408710+0000","last_unstale":"2024-04-02T16:22:07.434308+0000","last_undegraded":"2024-04-02T16:22:07.434308+0000","last_fullsized":"2024-04-02T16:22:07.434308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434473+0000","last_change":"2024-04-02T16:22:00.389406+0000","last_active":"2024-04-02T16:22:07.434473+0000","last_peered":"2024-04-02T16:22:07.434473+0000","last_clean":"2024-04-02T16:22:07.434473+0000","last_became_active":"2024-04-02T16:22:00.389107+0000","last_became_peered":"2024-04-02T16:22:00.389107+0000","last_unstale":"2024-04-02T16:22:07.434473+0000","last_undegraded":"2024-04-02T16:22:07.434473+0000","last_fullsized":"2024-04-02T16:22:07.434473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428740+0000","last_change":"2024-04-02T16:22:05.816005+0000","last_active":"2024-04-02T16:22:07.428740+0000","last_peered":"2024-04-02T16:22:07.428740+0000","last_clean":"2024-04-02T16:22:07.428740+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.428740+0000","last_undegraded":"2024-04-02T16:22:07.428740+0000","last_fullsized":"2024-04-02T16:22:07.428740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:45.290851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434745+0000","last_change":"2024-04-02T16:22:01.412447+0000","last_active":"2024-04-02T16:22:07.434745+0000","last_peered":"2024-04-02T16:22:07.434745+0000","last_clean":"2024-04-02T16:22:07.434745+0000","last_became_active":"2024-04-02T16:22:01.411677+0000","last_became_peered":"2024-04-02T16:22:01.411677+0000","last_unstale":"2024-04-02T16:22:07.434745+0000","last_undegraded":"2024-04-02T16:22:07.434745+0000","last_fullsized":"2024-04-02T16:22:07.434745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:41:01.979121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:00.389887+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:00.389287+0000","last_became_peered":"2024-04-02T16:22:00.389287+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428399+0000","last_change":"2024-04-02T16:22:05.815236+0000","last_active":"2024-04-02T16:22:07.428399+0000","last_peered":"2024-04-02T16:22:07.428399+0000","last_clean":"2024-04-02T16:22:07.428399+0000","last_became_active":"2024-04-02T16:22:05.814782+0000","last_became_peered":"2024-04-02T16:22:05.814782+0000","last_unstale":"2024-04-02T16:22:07.428399+0000","last_undegraded":"2024-04-02T16:22:07.428399+0000","last_fullsized":"2024-04-02T16:22:07.428399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:44.247231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434209+0000","last_change":"2024-04-02T16:22:01.401164+0000","last_active":"2024-04-02T16:22:07.434209+0000","last_peered":"2024-04-02T16:22:07.434209+0000","last_clean":"2024-04-02T16:22:07.434209+0000","last_became_active":"2024-04-02T16:22:01.400880+0000","last_became_peered":"2024-04-02T16:22:01.400880+0000","last_unstale":"2024-04-02T16:22:07.434209+0000","last_undegraded":"2024-04-02T16:22:07.434209+0000","last_fullsized":"2024-04-02T16:22:07.434209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435570+0000","last_change":"2024-04-02T16:22:00.386660+0000","last_active":"2024-04-02T16:22:07.435570+0000","last_peered":"2024-04-02T16:22:07.435570+0000","last_clean":"2024-04-02T16:22:07.435570+0000","last_became_active":"2024-04-02T16:22:00.386370+0000","last_became_peered":"2024-04-02T16:22:00.386370+0000","last_unstale":"2024-04-02T16:22:07.435570+0000","last_undegraded":"2024-04-02T16:22:07.435570+0000","last_fullsized":"2024-04-02T16:22:07.435570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:59:36.122507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430760+0000","last_change":"2024-04-02T16:22:05.816956+0000","last_active":"2024-04-02T16:22:07.430760+0000","last_peered":"2024-04-02T16:22:07.430760+0000","last_clean":"2024-04-02T16:22:07.430760+0000","last_became_active":"2024-04-02T16:22:05.816429+0000","last_became_peered":"2024-04-02T16:22:05.816429+0000","last_unstale":"2024-04-02T16:22:07.430760+0000","last_undegraded":"2024-04-02T16:22:07.430760+0000","last_fullsized":"2024-04-02T16:22:07.430760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:20:35.208462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434384+0000","last_change":"2024-04-02T16:22:01.405624+0000","last_active":"2024-04-02T16:22:07.434384+0000","last_peered":"2024-04-02T16:22:07.434384+0000","last_clean":"2024-04-02T16:22:07.434384+0000","last_became_active":"2024-04-02T16:22:01.405347+0000","last_became_peered":"2024-04-02T16:22:01.405347+0000","last_unstale":"2024-04-02T16:22:07.434384+0000","last_undegraded":"2024-04-02T16:22:07.434384+0000","last_fullsized":"2024-04-02T16:22:07.434384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:10.942808+0000","last_change":"2024-04-02T16:22:00.391986+0000","last_active":"2024-04-02T16:22:10.942808+0000","last_peered":"2024-04-02T16:22:10.942808+0000","last_clean":"2024-04-02T16:22:10.942808+0000","last_became_active":"2024-04-02T16:22:00.391681+0000","last_became_peered":"2024-04-02T16:22:00.391681+0000","last_unstale":"2024-04-02T16:22:10.942808+0000","last_undegraded":"2024-04-02T16:22:10.942808+0000","last_fullsized":"2024-04-02T16:22:10.942808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:49.923688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434848+0000","last_change":"2024-04-02T16:22:05.446704+0000","last_active":"2024-04-02T16:22:07.434848+0000","last_peered":"2024-04-02T16:22:07.434848+0000","last_clean":"2024-04-02T16:22:07.434848+0000","last_became_active":"2024-04-02T16:22:05.446150+0000","last_became_peered":"2024-04-02T16:22:05.446150+0000","last_unstale":"2024-04-02T16:22:07.434848+0000","last_undegraded":"2024-04-02T16:22:07.434848+0000","last_fullsized":"2024-04-02T16:22:07.434848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:24:01.645980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429828+0000","last_change":"2024-04-02T16:22:01.400842+0000","last_active":"2024-04-02T16:22:07.429828+0000","last_peered":"2024-04-02T16:22:07.429828+0000","last_clean":"2024-04-02T16:22:07.429828+0000","last_became_active":"2024-04-02T16:22:01.400565+0000","last_became_peered":"2024-04-02T16:22:01.400565+0000","last_unstale":"2024-04-02T16:22:07.429828+0000","last_undegraded":"2024-04-02T16:22:07.429828+0000","last_fullsized":"2024-04-02T16:22:07.429828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434908+0000","last_change":"2024-04-02T16:22:00.394135+0000","last_active":"2024-04-02T16:22:07.434908+0000","last_peered":"2024-04-02T16:22:07.434908+0000","last_clean":"2024-04-02T16:22:07.434908+0000","last_became_active":"2024-04-02T16:22:00.393846+0000","last_became_peered":"2024-04-02T16:22:00.393846+0000","last_unstale":"2024-04-02T16:22:07.434908+0000","last_undegraded":"2024-04-02T16:22:07.434908+0000","last_fullsized":"2024-04-02T16:22:07.434908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:32.926874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429907+0000","last_change":"2024-04-02T16:22:05.439118+0000","last_active":"2024-04-02T16:22:07.429907+0000","last_peered":"2024-04-02T16:22:07.429907+0000","last_clean":"2024-04-02T16:22:07.429907+0000","last_became_active":"2024-04-02T16:22:05.438557+0000","last_became_peered":"2024-04-02T16:22:05.438557+0000","last_unstale":"2024-04-02T16:22:07.429907+0000","last_undegraded":"2024-04-02T16:22:07.429907+0000","last_fullsized":"2024-04-02T16:22:07.429907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434885+0000","last_change":"2024-04-02T16:22:01.413531+0000","last_active":"2024-04-02T16:22:07.434885+0000","last_peered":"2024-04-02T16:22:07.434885+0000","last_clean":"2024-04-02T16:22:07.434885+0000","last_became_active":"2024-04-02T16:22:01.413229+0000","last_became_peered":"2024-04-02T16:22:01.413229+0000","last_unstale":"2024-04-02T16:22:07.434885+0000","last_undegraded":"2024-04-02T16:22:07.434885+0000","last_fullsized":"2024-04-02T16:22:07.434885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430736+0000","last_change":"2024-04-02T16:22:00.388772+0000","last_active":"2024-04-02T16:22:07.430736+0000","last_peered":"2024-04-02T16:22:07.430736+0000","last_clean":"2024-04-02T16:22:07.430736+0000","last_became_active":"2024-04-02T16:22:00.388500+0000","last_became_peered":"2024-04-02T16:22:00.388500+0000","last_unstale":"2024-04-02T16:22:07.430736+0000","last_undegraded":"2024-04-02T16:22:07.430736+0000","last_fullsized":"2024-04-02T16:22:07.430736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:00.316230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427878+0000","last_change":"2024-04-02T16:22:05.441542+0000","last_active":"2024-04-02T16:22:07.427878+0000","last_peered":"2024-04-02T16:22:07.427878+0000","last_clean":"2024-04-02T16:22:07.427878+0000","last_became_active":"2024-04-02T16:22:05.441134+0000","last_became_peered":"2024-04-02T16:22:05.441134+0000","last_unstale":"2024-04-02T16:22:07.427878+0000","last_undegraded":"2024-04-02T16:22:07.427878+0000","last_fullsized":"2024-04-02T16:22:07.427878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:02:44.237629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430701+0000","last_change":"2024-04-02T16:22:01.401315+0000","last_active":"2024-04-02T16:22:07.430701+0000","last_peered":"2024-04-02T16:22:07.430701+0000","last_clean":"2024-04-02T16:22:07.430701+0000","last_became_active":"2024-04-02T16:22:01.401023+0000","last_became_peered":"2024-04-02T16:22:01.401023+0000","last_unstale":"2024-04-02T16:22:07.430701+0000","last_undegraded":"2024-04-02T16:22:07.430701+0000","last_fullsized":"2024-04-02T16:22:07.430701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:18:04.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435375+0000","last_change":"2024-04-02T16:22:00.394082+0000","last_active":"2024-04-02T16:22:07.435375+0000","last_peered":"2024-04-02T16:22:07.435375+0000","last_clean":"2024-04-02T16:22:07.435375+0000","last_became_active":"2024-04-02T16:22:00.393793+0000","last_became_peered":"2024-04-02T16:22:00.393793+0000","last_unstale":"2024-04-02T16:22:07.435375+0000","last_undegraded":"2024-04-02T16:22:07.435375+0000","last_fullsized":"2024-04-02T16:22:07.435375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:52.296800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427851+0000","last_change":"2024-04-02T16:22:05.817432+0000","last_active":"2024-04-02T16:22:07.427851+0000","last_peered":"2024-04-02T16:22:07.427851+0000","last_clean":"2024-04-02T16:22:07.427851+0000","last_became_active":"2024-04-02T16:22:05.817018+0000","last_became_peered":"2024-04-02T16:22:05.817018+0000","last_unstale":"2024-04-02T16:22:07.427851+0000","last_undegraded":"2024-04-02T16:22:07.427851+0000","last_fullsized":"2024-04-02T16:22:07.427851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:52:22.886852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434205+0000","last_change":"2024-04-02T16:22:05.815342+0000","last_active":"2024-04-02T16:22:07.434205+0000","last_peered":"2024-04-02T16:22:07.434205+0000","last_clean":"2024-04-02T16:22:07.434205+0000","last_became_active":"2024-04-02T16:22:05.814977+0000","last_became_peered":"2024-04-02T16:22:05.814977+0000","last_unstale":"2024-04-02T16:22:07.434205+0000","last_undegraded":"2024-04-02T16:22:07.434205+0000","last_fullsized":"2024-04-02T16:22:07.434205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:16.062885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434398+0000","last_change":"2024-04-02T16:22:01.400394+0000","last_active":"2024-04-02T16:22:07.434398+0000","last_peered":"2024-04-02T16:22:07.434398+0000","last_clean":"2024-04-02T16:22:07.434398+0000","last_became_active":"2024-04-02T16:22:01.400000+0000","last_became_peered":"2024-04-02T16:22:01.400000+0000","last_unstale":"2024-04-02T16:22:07.434398+0000","last_undegraded":"2024-04-02T16:22:07.434398+0000","last_fullsized":"2024-04-02T16:22:07.434398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430602+0000","last_change":"2024-04-02T16:22:00.380328+0000","last_active":"2024-04-02T16:22:07.430602+0000","last_peered":"2024-04-02T16:22:07.430602+0000","last_clean":"2024-04-02T16:22:07.430602+0000","last_became_active":"2024-04-02T16:22:00.379294+0000","last_became_peered":"2024-04-02T16:22:00.379294+0000","last_unstale":"2024-04-02T16:22:07.430602+0000","last_undegraded":"2024-04-02T16:22:07.430602+0000","last_fullsized":"2024-04-02T16:22:07.430602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:09:25.659439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435187+0000","last_change":"2024-04-02T16:22:05.444408+0000","last_active":"2024-04-02T16:22:07.435187+0000","last_peered":"2024-04-02T16:22:07.435187+0000","last_clean":"2024-04-02T16:22:07.435187+0000","last_became_active":"2024-04-02T16:22:05.444100+0000","last_became_peered":"2024-04-02T16:22:05.444100+0000","last_unstale":"2024-04-02T16:22:07.435187+0000","last_undegraded":"2024-04-02T16:22:07.435187+0000","last_fullsized":"2024-04-02T16:22:07.435187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:09.526099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427872+0000","last_change":"2024-04-02T16:22:01.409456+0000","last_active":"2024-04-02T16:22:07.427872+0000","last_peered":"2024-04-02T16:22:07.427872+0000","last_clean":"2024-04-02T16:22:07.427872+0000","last_became_active":"2024-04-02T16:22:01.408824+0000","last_became_peered":"2024-04-02T16:22:01.408824+0000","last_unstale":"2024-04-02T16:22:07.427872+0000","last_undegraded":"2024-04-02T16:22:07.427872+0000","last_fullsized":"2024-04-02T16:22:07.427872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427970+0000","last_change":"2024-04-02T16:22:00.390487+0000","last_active":"2024-04-02T16:22:07.427970+0000","last_peered":"2024-04-02T16:22:07.427970+0000","last_clean":"2024-04-02T16:22:07.427970+0000","last_became_active":"2024-04-02T16:22:00.390198+0000","last_became_peered":"2024-04-02T16:22:00.390198+0000","last_unstale":"2024-04-02T16:22:07.427970+0000","last_undegraded":"2024-04-02T16:22:07.427970+0000","last_fullsized":"2024-04-02T16:22:07.427970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:02.864324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434683+0000","last_change":"2024-04-02T16:22:00.388624+0000","last_active":"2024-04-02T16:22:07.434683+0000","last_peered":"2024-04-02T16:22:07.434683+0000","last_clean":"2024-04-02T16:22:07.434683+0000","last_became_active":"2024-04-02T16:22:00.388351+0000","last_became_peered":"2024-04-02T16:22:00.388351+0000","last_unstale":"2024-04-02T16:22:07.434683+0000","last_undegraded":"2024-04-02T16:22:07.434683+0000","last_fullsized":"2024-04-02T16:22:07.434683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:06:26.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435511+0000","last_change":"2024-04-02T16:22:01.403982+0000","last_active":"2024-04-02T16:22:07.435511+0000","last_peered":"2024-04-02T16:22:07.435511+0000","last_clean":"2024-04-02T16:22:07.435511+0000","last_became_active":"2024-04-02T16:22:01.403057+0000","last_became_peered":"2024-04-02T16:22:01.403057+0000","last_unstale":"2024-04-02T16:22:07.435511+0000","last_undegraded":"2024-04-02T16:22:07.435511+0000","last_fullsized":"2024-04-02T16:22:07.435511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434167+0000","last_change":"2024-04-02T16:22:05.443373+0000","last_active":"2024-04-02T16:22:07.434167+0000","last_peered":"2024-04-02T16:22:07.434167+0000","last_clean":"2024-04-02T16:22:07.434167+0000","last_became_active":"2024-04-02T16:22:05.442934+0000","last_became_peered":"2024-04-02T16:22:05.442934+0000","last_unstale":"2024-04-02T16:22:07.434167+0000","last_undegraded":"2024-04-02T16:22:07.434167+0000","last_fullsized":"2024-04-02T16:22:07.434167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:58:49.857086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428264+0000","last_change":"2024-04-02T16:22:00.391680+0000","last_active":"2024-04-02T16:22:07.428264+0000","last_peered":"2024-04-02T16:22:07.428264+0000","last_clean":"2024-04-02T16:22:07.428264+0000","last_became_active":"2024-04-02T16:22:00.391402+0000","last_became_peered":"2024-04-02T16:22:00.391402+0000","last_unstale":"2024-04-02T16:22:07.428264+0000","last_undegraded":"2024-04-02T16:22:07.428264+0000","last_fullsized":"2024-04-02T16:22:07.428264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:59.364392+0000","last_clean_scrub_stamp":"2024-04-02T16:21:59.364392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:21.986541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429224+0000","last_change":"2024-04-02T16:22:01.398824+0000","last_active":"2024-04-02T16:22:07.429224+0000","last_peered":"2024-04-02T16:22:07.429224+0000","last_clean":"2024-04-02T16:22:07.429224+0000","last_became_active":"2024-04-02T16:22:01.398529+0000","last_became_peered":"2024-04-02T16:22:01.398529+0000","last_unstale":"2024-04-02T16:22:07.429224+0000","last_undegraded":"2024-04-02T16:22:07.429224+0000","last_fullsized":"2024-04-02T16:22:07.429224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:42:01.320427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428686+0000","last_change":"2024-04-02T16:22:05.440423+0000","last_active":"2024-04-02T16:22:07.428686+0000","last_peered":"2024-04-02T16:22:07.428686+0000","last_clean":"2024-04-02T16:22:07.428686+0000","last_became_active":"2024-04-02T16:22:05.439993+0000","last_became_peered":"2024-04-02T16:22:05.439993+0000","last_unstale":"2024-04-02T16:22:07.428686+0000","last_undegraded":"2024-04-02T16:22:07.428686+0000","last_fullsized":"2024-04-02T16:22:07.428686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:04.415636+0000","last_clean_scrub_stamp":"2024-04-02T16:22:04.415636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:17.805386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434177+0000","last_change":"2024-04-02T16:22:01.408902+0000","last_active":"2024-04-02T16:22:07.434177+0000","last_peered":"2024-04-02T16:22:07.434177+0000","last_clean":"2024-04-02T16:22:07.434177+0000","last_became_active":"2024-04-02T16:22:01.408562+0000","last_became_peered":"2024-04-02T16:22:01.408562+0000","last_unstale":"2024-04-02T16:22:07.434177+0000","last_undegraded":"2024-04-02T16:22:07.434177+0000","last_fullsized":"2024-04-02T16:22:07.434177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433873+0000","last_change":"2024-04-02T16:22:01.404001+0000","last_active":"2024-04-02T16:22:07.433873+0000","last_peered":"2024-04-02T16:22:07.433873+0000","last_clean":"2024-04-02T16:22:07.433873+0000","last_became_active":"2024-04-02T16:22:01.403694+0000","last_became_peered":"2024-04-02T16:22:01.403694+0000","last_unstale":"2024-04-02T16:22:07.433873+0000","last_undegraded":"2024-04-02T16:22:07.433873+0000","last_fullsized":"2024-04-02T16:22:07.433873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429709+0000","last_change":"2024-04-02T16:22:01.407141+0000","last_active":"2024-04-02T16:22:07.429709+0000","last_peered":"2024-04-02T16:22:07.429709+0000","last_clean":"2024-04-02T16:22:07.429709+0000","last_became_active":"2024-04-02T16:22:01.403018+0000","last_became_peered":"2024-04-02T16:22:01.403018+0000","last_unstale":"2024-04-02T16:22:07.429709+0000","last_undegraded":"2024-04-02T16:22:07.429709+0000","last_fullsized":"2024-04-02T16:22:07.429709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:01:40.100919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435416+0000","last_change":"2024-04-02T16:22:01.403712+0000","last_active":"2024-04-02T16:22:07.435416+0000","last_peered":"2024-04-02T16:22:07.435416+0000","last_clean":"2024-04-02T16:22:07.435416+0000","last_became_active":"2024-04-02T16:22:01.402419+0000","last_became_peered":"2024-04-02T16:22:01.402419+0000","last_unstale":"2024-04-02T16:22:07.435416+0000","last_undegraded":"2024-04-02T16:22:07.435416+0000","last_fullsized":"2024-04-02T16:22:07.435416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:47.631501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428475+0000","last_change":"2024-04-02T16:22:01.411196+0000","last_active":"2024-04-02T16:22:07.428475+0000","last_peered":"2024-04-02T16:22:07.428475+0000","last_clean":"2024-04-02T16:22:07.428475+0000","last_became_active":"2024-04-02T16:22:01.410630+0000","last_became_peered":"2024-04-02T16:22:01.410630+0000","last_unstale":"2024-04-02T16:22:07.428475+0000","last_undegraded":"2024-04-02T16:22:07.428475+0000","last_fullsized":"2024-04-02T16:22:07.428475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434802+0000","last_change":"2024-04-02T16:22:01.401113+0000","last_active":"2024-04-02T16:22:07.434802+0000","last_peered":"2024-04-02T16:22:07.434802+0000","last_clean":"2024-04-02T16:22:07.434802+0000","last_became_active":"2024-04-02T16:22:01.400823+0000","last_became_peered":"2024-04-02T16:22:01.400823+0000","last_unstale":"2024-04-02T16:22:07.434802+0000","last_undegraded":"2024-04-02T16:22:07.434802+0000","last_fullsized":"2024-04-02T16:22:07.434802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435248+0000","last_change":"2024-04-02T16:22:01.401273+0000","last_active":"2024-04-02T16:22:07.435248+0000","last_peered":"2024-04-02T16:22:07.435248+0000","last_clean":"2024-04-02T16:22:07.435248+0000","last_became_active":"2024-04-02T16:22:01.400989+0000","last_became_peered":"2024-04-02T16:22:01.400989+0000","last_unstale":"2024-04-02T16:22:07.435248+0000","last_undegraded":"2024-04-02T16:22:07.435248+0000","last_fullsized":"2024-04-02T16:22:07.435248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428640+0000","last_change":"2024-04-02T16:22:01.410167+0000","last_active":"2024-04-02T16:22:07.428640+0000","last_peered":"2024-04-02T16:22:07.428640+0000","last_clean":"2024-04-02T16:22:07.428640+0000","last_became_active":"2024-04-02T16:22:01.409747+0000","last_became_peered":"2024-04-02T16:22:01.409747+0000","last_unstale":"2024-04-02T16:22:07.428640+0000","last_undegraded":"2024-04-02T16:22:07.428640+0000","last_fullsized":"2024-04-02T16:22:07.428640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430485+0000","last_change":"2024-04-02T16:22:01.404722+0000","last_active":"2024-04-02T16:22:07.430485+0000","last_peered":"2024-04-02T16:22:07.430485+0000","last_clean":"2024-04-02T16:22:07.430485+0000","last_became_active":"2024-04-02T16:22:01.403645+0000","last_became_peered":"2024-04-02T16:22:01.403645+0000","last_unstale":"2024-04-02T16:22:07.430485+0000","last_undegraded":"2024-04-02T16:22:07.430485+0000","last_fullsized":"2024-04-02T16:22:07.430485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427877+0000","last_change":"2024-04-02T16:22:01.410096+0000","last_active":"2024-04-02T16:22:07.427877+0000","last_peered":"2024-04-02T16:22:07.427877+0000","last_clean":"2024-04-02T16:22:07.427877+0000","last_became_active":"2024-04-02T16:22:01.409805+0000","last_became_peered":"2024-04-02T16:22:01.409805+0000","last_unstale":"2024-04-02T16:22:07.427877+0000","last_undegraded":"2024-04-02T16:22:07.427877+0000","last_fullsized":"2024-04-02T16:22:07.427877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429582+0000","last_change":"2024-04-02T16:22:01.410138+0000","last_active":"2024-04-02T16:22:07.429582+0000","last_peered":"2024-04-02T16:22:07.429582+0000","last_clean":"2024-04-02T16:22:07.429582+0000","last_became_active":"2024-04-02T16:22:01.403179+0000","last_became_peered":"2024-04-02T16:22:01.403179+0000","last_unstale":"2024-04-02T16:22:07.429582+0000","last_undegraded":"2024-04-02T16:22:07.429582+0000","last_fullsized":"2024-04-02T16:22:07.429582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:45.121884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435300+0000","last_change":"2024-04-02T16:22:01.403457+0000","last_active":"2024-04-02T16:22:07.435300+0000","last_peered":"2024-04-02T16:22:07.435300+0000","last_clean":"2024-04-02T16:22:07.435300+0000","last_became_active":"2024-04-02T16:22:01.402335+0000","last_became_peered":"2024-04-02T16:22:01.402335+0000","last_unstale":"2024-04-02T16:22:07.435300+0000","last_undegraded":"2024-04-02T16:22:07.435300+0000","last_fullsized":"2024-04-02T16:22:07.435300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:55.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434948+0000","last_change":"2024-04-02T16:22:01.406544+0000","last_active":"2024-04-02T16:22:07.434948+0000","last_peered":"2024-04-02T16:22:07.434948+0000","last_clean":"2024-04-02T16:22:07.434948+0000","last_became_active":"2024-04-02T16:22:01.406205+0000","last_became_peered":"2024-04-02T16:22:01.406205+0000","last_unstale":"2024-04-02T16:22:07.434948+0000","last_undegraded":"2024-04-02T16:22:07.434948+0000","last_fullsized":"2024-04-02T16:22:07.434948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435085+0000","last_change":"2024-04-02T16:22:01.401972+0000","last_active":"2024-04-02T16:22:07.435085+0000","last_peered":"2024-04-02T16:22:07.435085+0000","last_clean":"2024-04-02T16:22:07.435085+0000","last_became_active":"2024-04-02T16:22:01.401721+0000","last_became_peered":"2024-04-02T16:22:01.401721+0000","last_unstale":"2024-04-02T16:22:07.435085+0000","last_undegraded":"2024-04-02T16:22:07.435085+0000","last_fullsized":"2024-04-02T16:22:07.435085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:05.479052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.433925+0000","last_change":"2024-04-02T16:22:01.406917+0000","last_active":"2024-04-02T16:22:07.433925+0000","last_peered":"2024-04-02T16:22:07.433925+0000","last_clean":"2024-04-02T16:22:07.433925+0000","last_became_active":"2024-04-02T16:22:01.406622+0000","last_became_peered":"2024-04-02T16:22:01.406622+0000","last_unstale":"2024-04-02T16:22:07.433925+0000","last_undegraded":"2024-04-02T16:22:07.433925+0000","last_fullsized":"2024-04-02T16:22:07.433925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:45.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430970+0000","last_change":"2024-04-02T16:22:01.403891+0000","last_active":"2024-04-02T16:22:07.430970+0000","last_peered":"2024-04-02T16:22:07.430970+0000","last_clean":"2024-04-02T16:22:07.430970+0000","last_became_active":"2024-04-02T16:22:01.403250+0000","last_became_peered":"2024-04-02T16:22:01.403250+0000","last_unstale":"2024-04-02T16:22:07.430970+0000","last_undegraded":"2024-04-02T16:22:07.430970+0000","last_fullsized":"2024-04-02T16:22:07.430970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:48:44.990696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430459+0000","last_change":"2024-04-02T16:22:01.410686+0000","last_active":"2024-04-02T16:22:07.430459+0000","last_peered":"2024-04-02T16:22:07.430459+0000","last_clean":"2024-04-02T16:22:07.430459+0000","last_became_active":"2024-04-02T16:22:01.410398+0000","last_became_peered":"2024-04-02T16:22:01.410398+0000","last_unstale":"2024-04-02T16:22:07.430459+0000","last_undegraded":"2024-04-02T16:22:07.430459+0000","last_fullsized":"2024-04-02T16:22:07.430459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:27:20.054917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434490+0000","last_change":"2024-04-02T16:22:01.406912+0000","last_active":"2024-04-02T16:22:07.434490+0000","last_peered":"2024-04-02T16:22:07.434490+0000","last_clean":"2024-04-02T16:22:07.434490+0000","last_became_active":"2024-04-02T16:22:01.406626+0000","last_became_peered":"2024-04-02T16:22:01.406626+0000","last_unstale":"2024-04-02T16:22:07.434490+0000","last_undegraded":"2024-04-02T16:22:07.434490+0000","last_fullsized":"2024-04-02T16:22:07.434490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434095+0000","last_change":"2024-04-02T16:22:01.397661+0000","last_active":"2024-04-02T16:22:07.434095+0000","last_peered":"2024-04-02T16:22:07.434095+0000","last_clean":"2024-04-02T16:22:07.434095+0000","last_became_active":"2024-04-02T16:22:01.397335+0000","last_became_peered":"2024-04-02T16:22:01.397335+0000","last_unstale":"2024-04-02T16:22:07.434095+0000","last_undegraded":"2024-04-02T16:22:07.434095+0000","last_fullsized":"2024-04-02T16:22:07.434095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.428147+0000","last_change":"2024-04-02T16:22:01.409121+0000","last_active":"2024-04-02T16:22:07.428147+0000","last_peered":"2024-04-02T16:22:07.428147+0000","last_clean":"2024-04-02T16:22:07.428147+0000","last_became_active":"2024-04-02T16:22:01.408765+0000","last_became_peered":"2024-04-02T16:22:01.408765+0000","last_unstale":"2024-04-02T16:22:07.428147+0000","last_undegraded":"2024-04-02T16:22:07.428147+0000","last_fullsized":"2024-04-02T16:22:07.428147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429788+0000","last_change":"2024-04-02T16:22:01.407108+0000","last_active":"2024-04-02T16:22:07.429788+0000","last_peered":"2024-04-02T16:22:07.429788+0000","last_clean":"2024-04-02T16:22:07.429788+0000","last_became_active":"2024-04-02T16:22:01.406836+0000","last_became_peered":"2024-04-02T16:22:01.406836+0000","last_unstale":"2024-04-02T16:22:07.429788+0000","last_undegraded":"2024-04-02T16:22:07.429788+0000","last_fullsized":"2024-04-02T16:22:07.429788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:33.931071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434905+0000","last_change":"2024-04-02T16:22:01.414174+0000","last_active":"2024-04-02T16:22:07.434905+0000","last_peered":"2024-04-02T16:22:07.434905+0000","last_clean":"2024-04-02T16:22:07.434905+0000","last_became_active":"2024-04-02T16:22:01.413903+0000","last_became_peered":"2024-04-02T16:22:01.413903+0000","last_unstale":"2024-04-02T16:22:07.434905+0000","last_undegraded":"2024-04-02T16:22:07.434905+0000","last_fullsized":"2024-04-02T16:22:07.434905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:51.907352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435234+0000","last_change":"2024-04-02T16:22:01.413533+0000","last_active":"2024-04-02T16:22:07.435234+0000","last_peered":"2024-04-02T16:22:07.435234+0000","last_clean":"2024-04-02T16:22:07.435234+0000","last_became_active":"2024-04-02T16:22:01.413266+0000","last_became_peered":"2024-04-02T16:22:01.413266+0000","last_unstale":"2024-04-02T16:22:07.435234+0000","last_undegraded":"2024-04-02T16:22:07.435234+0000","last_fullsized":"2024-04-02T16:22:07.435234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435468+0000","last_change":"2024-04-02T16:22:01.412176+0000","last_active":"2024-04-02T16:22:07.435468+0000","last_peered":"2024-04-02T16:22:07.435468+0000","last_clean":"2024-04-02T16:22:07.435468+0000","last_became_active":"2024-04-02T16:22:01.411793+0000","last_became_peered":"2024-04-02T16:22:01.411793+0000","last_unstale":"2024-04-02T16:22:07.435468+0000","last_undegraded":"2024-04-02T16:22:07.435468+0000","last_fullsized":"2024-04-02T16:22:07.435468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:12.015838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.427771+0000","last_change":"2024-04-02T16:22:01.410605+0000","last_active":"2024-04-02T16:22:07.427771+0000","last_peered":"2024-04-02T16:22:07.427771+0000","last_clean":"2024-04-02T16:22:07.427771+0000","last_became_active":"2024-04-02T16:22:01.410237+0000","last_became_peered":"2024-04-02T16:22:01.410237+0000","last_unstale":"2024-04-02T16:22:07.427771+0000","last_undegraded":"2024-04-02T16:22:07.427771+0000","last_fullsized":"2024-04-02T16:22:07.427771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:01.421830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434564+0000","last_change":"2024-04-02T16:22:01.404165+0000","last_active":"2024-04-02T16:22:07.434564+0000","last_peered":"2024-04-02T16:22:07.434564+0000","last_clean":"2024-04-02T16:22:07.434564+0000","last_became_active":"2024-04-02T16:22:01.403907+0000","last_became_peered":"2024-04-02T16:22:01.403907+0000","last_unstale":"2024-04-02T16:22:07.434564+0000","last_undegraded":"2024-04-02T16:22:07.434564+0000","last_fullsized":"2024-04-02T16:22:07.434564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:15.099148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.429480+0000","last_change":"2024-04-02T16:22:01.409810+0000","last_active":"2024-04-02T16:22:07.429480+0000","last_peered":"2024-04-02T16:22:07.429480+0000","last_clean":"2024-04-02T16:22:07.429480+0000","last_became_active":"2024-04-02T16:22:01.406447+0000","last_became_peered":"2024-04-02T16:22:01.406447+0000","last_unstale":"2024-04-02T16:22:07.429480+0000","last_undegraded":"2024-04-02T16:22:07.429480+0000","last_fullsized":"2024-04-02T16:22:07.429480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:32:31.833931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434648+0000","last_change":"2024-04-02T16:22:01.401898+0000","last_active":"2024-04-02T16:22:07.434648+0000","last_peered":"2024-04-02T16:22:07.434648+0000","last_clean":"2024-04-02T16:22:07.434648+0000","last_became_active":"2024-04-02T16:22:01.401568+0000","last_became_peered":"2024-04-02T16:22:01.401568+0000","last_unstale":"2024-04-02T16:22:07.434648+0000","last_undegraded":"2024-04-02T16:22:07.434648+0000","last_fullsized":"2024-04-02T16:22:07.434648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:11:08.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.430221+0000","last_change":"2024-04-02T16:22:01.410551+0000","last_active":"2024-04-02T16:22:07.430221+0000","last_peered":"2024-04-02T16:22:07.430221+0000","last_clean":"2024-04-02T16:22:07.430221+0000","last_became_active":"2024-04-02T16:22:01.409045+0000","last_became_peered":"2024-04-02T16:22:01.409045+0000","last_unstale":"2024-04-02T16:22:07.430221+0000","last_undegraded":"2024-04-02T16:22:07.430221+0000","last_fullsized":"2024-04-02T16:22:07.430221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:26:36.255961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.434517+0000","last_change":"2024-04-02T16:22:01.398994+0000","last_active":"2024-04-02T16:22:07.434517+0000","last_peered":"2024-04-02T16:22:07.434517+0000","last_clean":"2024-04-02T16:22:07.434517+0000","last_became_active":"2024-04-02T16:22:01.398698+0000","last_became_peered":"2024-04-02T16:22:01.398698+0000","last_unstale":"2024-04-02T16:22:07.434517+0000","last_undegraded":"2024-04-02T16:22:07.434517+0000","last_fullsized":"2024-04-02T16:22:07.434517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:12:14.142237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435172+0000","last_change":"2024-04-02T16:22:01.406275+0000","last_active":"2024-04-02T16:22:07.435172+0000","last_peered":"2024-04-02T16:22:07.435172+0000","last_clean":"2024-04-02T16:22:07.435172+0000","last_became_active":"2024-04-02T16:22:01.405936+0000","last_became_peered":"2024-04-02T16:22:01.405936+0000","last_unstale":"2024-04-02T16:22:07.435172+0000","last_undegraded":"2024-04-02T16:22:07.435172+0000","last_fullsized":"2024-04-02T16:22:07.435172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:54.215893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-02T16:22:07.435388+0000","last_change":"2024-04-02T16:22:01.407470+0000","last_active":"2024-04-02T16:22:07.435388+0000","last_peered":"2024-04-02T16:22:07.435388+0000","last_clean":"2024-04-02T16:22:07.435388+0000","last_became_active":"2024-04-02T16:22:01.407209+0000","last_became_peered":"2024-04-02T16:22:01.407209+0000","last_unstale":"2024-04-02T16:22:07.435388+0000","last_undegraded":"2024-04-02T16:22:07.435388+0000","last_fullsized":"2024-04-02T16:22:07.435388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:22:00.368589+0000","last_clean_scrub_stamp":"2024-04-02T16:22:00.368589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:31.303906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:22:29.318 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-02T16:22:29.318 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T16:22:29.318 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-02T16:22:29.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-02T16:22:29.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.453+0000 7fbb0b088640 1 Processor -- start 2024-04-02T16:22:29.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.453+0000 7fbb0b088640 1 -- start start 2024-04-02T16:22:29.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb0b088640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb040b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb0b088640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbb040b4760 0x7fbb040d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb0b088640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb040b4c70 con 0x7fbb040d2cb0 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb0b088640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fbb040b5880 con 0x7fbb040b4760 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb0b088640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040b4f50 con 0x7fbb0415e650 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb03fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbb040b4760 0x7fbb040d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb09625640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fbb040d2cb0 0x7fbb041622e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55104/0 (socket says 172.21.15.50:55104) 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.454+0000 7fbb09625640 1 -- 172.21.15.50:0/1524121576 learned_addr learned my addr 172.21.15.50:0/1524121576 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.455+0000 7fbb08e24640 1 --2- 172.21.15.50:0/1524121576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb040b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.455+0000 7fbb03fff640 1 -- 172.21.15.50:0/1524121576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 msgr2=0x7fbb040b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.455+0000 7fbb03fff640 1 --2- 172.21.15.50:0/1524121576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb040b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.455+0000 7fbb03fff640 1 -- 172.21.15.50:0/1524121576 >> v1:172.21.15.50:6789/0 conn(0x7fbb040d2cb0 legacy=0x7fbb041622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.455+0000 7fbb037fe640 1 -- 172.21.15.50:0/1524121576 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392741847 0 0) 0x7fbb040b4c70 con 0x7fbb040d2cb0 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.455+0000 7fbb03fff640 1 -- 172.21.15.50:0/1524121576 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb040b5230 con 0x7fbb040b4760 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.455+0000 7fbb03fff640 1 --2- 172.21.15.50:0/1524121576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040b4760 0x7fbb040d2770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009d90 tx=0x7fbaf402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237f93a18b6cf8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb037fe640 1 -- 172.21.15.50:0/1524121576 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4039070 con 0x7fbb040b4760 2024-04-02T16:22:29.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb037fe640 1 -- 172.21.15.50:0/1524121576 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf402cc50 con 0x7fbb040b4760 2024-04-02T16:22:29.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb037fe640 1 -- 172.21.15.50:0/1524121576 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4034440 con 0x7fbb040b4760 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 -- 172.21.15.50:0/1524121576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040b4760 msgr2=0x7fbb040d2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 --2- 172.21.15.50:0/1524121576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040b4760 0x7fbb040d2770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009d90 tx=0x7fbaf402bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 -- 172.21.15.50:0/1524121576 shutdown_connections 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 --2- 172.21.15.50:0/1524121576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040b4760 0x7fbb040d2770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 --2- 172.21.15.50:0/1524121576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb040b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 -- 172.21.15.50:0/1524121576 >> 172.21.15.50:0/1524121576 conn(0x7fbb0409d580 msgr2=0x7fbb040d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 -- 172.21.15.50:0/1524121576 shutdown_connections 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.456+0000 7fbb0b088640 1 -- 172.21.15.50:0/1524121576 wait complete. 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.457+0000 7fbb0b088640 1 Processor -- start 2024-04-02T16:22:29.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.457+0000 7fbb0b088640 1 -- start start 2024-04-02T16:22:29.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb0b088640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb040b4760 0x7fbb040a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb0b088640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040d2cb0 0x7fbb041f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb0b088640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb041f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb0b088640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0415ea30 con 0x7fbb040d2cb0 2024-04-02T16:22:29.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb0b088640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040d2270 con 0x7fbb040b4760 2024-04-02T16:22:29.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb0b088640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0415ebb0 con 0x7fbb0415e650 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040d2cb0 0x7fbb041f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040d2cb0 0x7fbb041f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45396/0 (socket says 172.21.15.50:45396) 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb08e24640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb040b4760 0x7fbb040a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb03fff640 1 -- 172.21.15.50:0/2070710869 learned_addr learned my addr 172.21.15.50:0/2070710869 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.458+0000 7fbb09625640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb041f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb03fff640 1 -- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 msgr2=0x7fbb041f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb03fff640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb041f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb03fff640 1 -- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb040b4760 msgr2=0x7fbb040a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb03fff640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb040b4760 0x7fbb040a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb03fff640 1 -- 172.21.15.50:0/2070710869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041f5340 con 0x7fbb040d2cb0 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb03fff640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040d2cb0 0x7fbb041f7640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4037040 tx=0x7fbaf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb017fa640 1 -- 172.21.15.50:0/2070710869 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4044020 con 0x7fbb040d2cb0 2024-04-02T16:22:29.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb017fa640 1 -- 172.21.15.50:0/2070710869 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf402ca50 con 0x7fbb040d2cb0 2024-04-02T16:22:29.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041f5620 con 0x7fbb040d2cb0 2024-04-02T16:22:29.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb017fa640 1 -- 172.21.15.50:0/2070710869 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4039040 con 0x7fbb040d2cb0 2024-04-02T16:22:29.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.459+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041f5be0 con 0x7fbb040d2cb0 2024-04-02T16:22:29.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.460+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb040756f0 con 0x7fbb040d2cb0 2024-04-02T16:22:29.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.461+0000 7fbb017fa640 1 -- 172.21.15.50:0/2070710869 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaf4047020 con 0x7fbb040d2cb0 2024-04-02T16:22:29.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.461+0000 7fbb017fa640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.461+0000 7fbb017fa640 1 -- 172.21.15.50:0/2070710869 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbaf40ae690 con 0x7fbb040d2cb0 2024-04-02T16:22:29.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.461+0000 7fbb08e24640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.462+0000 7fbb08e24640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb040b8be0 tx=0x7fbaec009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.465+0000 7fbb017fa640 1 -- 172.21.15.50:0/2070710869 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbaf404e030 con 0x7fbb040d2cb0 2024-04-02T16:22:29.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.689+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbb040d1aa0 con 0x7fbb040d2cb0 2024-04-02T16:22:29.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.690+0000 7fbb017fa640 1 -- 172.21.15.50:0/2070710869 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbaf402cc20 con 0x7fbb040d2cb0 2024-04-02T16:22:29.691 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:29.691 INFO:teuthology.orchestra.run.smithi050.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T16:22:29.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.691+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbae406c8f0 msgr2=0x7fbae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.691+0000 7fbb0b088640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb040b8be0 tx=0x7fbaec009190 comp rx=0 tx=0).stop 2024-04-02T16:22:29.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.691+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040d2cb0 msgr2=0x7fbb041f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.691+0000 7fbb0b088640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040d2cb0 0x7fbb041f7640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4037040 tx=0x7fbaf4004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:29.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 shutdown_connections 2024-04-02T16:22:29.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb0415e650 0x7fbb041f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb040d2cb0 0x7fbb041f7640 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 --2- 172.21.15.50:0/2070710869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb040b4760 0x7fbb040a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 >> 172.21.15.50:0/2070710869 conn(0x7fbb0409d580 msgr2=0x7fbb040d4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 shutdown_connections 2024-04-02T16:22:29.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.692+0000 7fbb0b088640 1 -- 172.21.15.50:0/2070710869 wait complete. 2024-04-02T16:22:29.703 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-02T16:22:29.704 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:29.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.804+0000 7f3844d1c640 1 Processor -- start 2024-04-02T16:22:29.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.804+0000 7f3844d1c640 1 -- start start 2024-04-02T16:22:29.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f3844d1c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f384006f1f0 0x7f3840169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f3844d1c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38400d0bd0 con 0x7f38400b37d0 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f3844d1c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38400d0eb0 con 0x7f384015e430 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f3844d1c640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f384006d9e0 con 0x7f384006f1f0 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f383ed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f384006f1f0 0x7f3840169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f383dd9b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f38400b37d0 0x7f38400cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55108/0 (socket says 172.21.15.50:55108) 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f383dd9b640 1 -- 172.21.15.50:0/539352695 learned_addr learned my addr 172.21.15.50:0/539352695 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f383ed9d640 1 -- 172.21.15.50:0/539352695 >> v1:172.21.15.188:6789/0 conn(0x7f384015e430 legacy=0x7f38400b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f383ed9d640 1 -- 172.21.15.50:0/539352695 >> v1:172.21.15.50:6789/0 conn(0x7f38400b37d0 legacy=0x7f38400cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.805+0000 7f383ed9d640 1 -- 172.21.15.50:0/539352695 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384006dcc0 con 0x7f384006f1f0 2024-04-02T16:22:29.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.806+0000 7f383ed9d640 1 --2- 172.21.15.50:0/539352695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f384006f1f0 0x7f3840169140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3830009d90 tx=0x7f383002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4799c35c616bdca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.806+0000 7f383d59a640 1 -- 172.21.15.50:0/539352695 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830039070 con 0x7f384006f1f0 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.806+0000 7f383d59a640 1 -- 172.21.15.50:0/539352695 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383002cc60 con 0x7f384006f1f0 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.806+0000 7f383d59a640 1 -- 172.21.15.50:0/539352695 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830034660 con 0x7f384006f1f0 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.807+0000 7f3844d1c640 1 -- 172.21.15.50:0/539352695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f384006f1f0 msgr2=0x7f3840169140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.807+0000 7f3844d1c640 1 --2- 172.21.15.50:0/539352695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f384006f1f0 0x7f3840169140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3830009d90 tx=0x7f383002beb0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.807+0000 7f3844d1c640 1 -- 172.21.15.50:0/539352695 shutdown_connections 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.807+0000 7f3844d1c640 1 --2- 172.21.15.50:0/539352695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f384006f1f0 0x7f3840169140 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.807+0000 7f3844d1c640 1 -- 172.21.15.50:0/539352695 >> 172.21.15.50:0/539352695 conn(0x7f38400b5620 msgr2=0x7f38400b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.807+0000 7f3844d1c640 1 -- 172.21.15.50:0/539352695 shutdown_connections 2024-04-02T16:22:29.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.807+0000 7f3844d1c640 1 -- 172.21.15.50:0/539352695 wait complete. 2024-04-02T16:22:29.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.808+0000 7f3844d1c640 1 Processor -- start 2024-04-02T16:22:29.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.808+0000 7f3844d1c640 1 -- start start 2024-04-02T16:22:29.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f3844d1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384006f1f0 0x7f3840162570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f3844d1c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38400b37d0 0x7f38400c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f3844d1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015e430 0x7f384014b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f3844d1c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38400d07a0 con 0x7f38400b37d0 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f3844d1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38400d0620 con 0x7f384015e430 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f3844d1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f38400d0920 con 0x7f384006f1f0 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38400b37d0 0x7f38400c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38400b37d0 0x7f38400c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45424/0 (socket says 172.21.15.50:45424) 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 -- 172.21.15.50:0/652319591 learned_addr learned my addr 172.21.15.50:0/652319591 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383ed9d640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015e430 0x7f384014b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383e59c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384006f1f0 0x7f3840162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 -- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384006f1f0 msgr2=0x7f3840162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384006f1f0 0x7f3840162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 -- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015e430 msgr2=0x7f384014b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015e430 0x7f384014b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383dd9b640 1 -- 172.21.15.50:0/652319591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38400c9e60 con 0x7f38400b37d0 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.809+0000 7f383ed9d640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015e430 0x7f384014b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.810+0000 7f383dd9b640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38400b37d0 0x7f38400c9480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f383400c4a0 tx=0x7f383400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.810+0000 7f382effd640 1 -- 172.21.15.50:0/652319591 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834017070 con 0x7f38400b37d0 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.810+0000 7f382effd640 1 -- 172.21.15.50:0/652319591 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3834005ce0 con 0x7f38400b37d0 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.810+0000 7f382effd640 1 -- 172.21.15.50:0/652319591 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834007270 con 0x7f38400b37d0 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.810+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38400ca140 con 0x7f38400b37d0 2024-04-02T16:22:29.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.810+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38400ca700 con 0x7f38400b37d0 2024-04-02T16:22:29.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.811+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38400756f0 con 0x7f38400b37d0 2024-04-02T16:22:29.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.812+0000 7f382effd640 1 -- 172.21.15.50:0/652319591 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f383401a070 con 0x7f38400b37d0 2024-04-02T16:22:29.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.812+0000 7f382effd640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f382006c8f0 0x7f382006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:29.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.812+0000 7f382effd640 1 -- 172.21.15.50:0/652319591 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f383408b0e0 con 0x7f38400b37d0 2024-04-02T16:22:29.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.812+0000 7f383e59c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f382006c8f0 0x7f382006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:29.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.813+0000 7f383e59c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f382006c8f0 0x7f382006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3838006fd0 tx=0x7f3838008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:29.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:29.817+0000 7f382effd640 1 -- 172.21.15.50:0/652319591 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3834058c10 con 0x7f38400b37d0 2024-04-02T16:22:30.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.002+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f384015d360 con 0x7f38400b37d0 2024-04-02T16:22:30.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.003+0000 7f382effd640 1 -- 172.21.15.50:0/652319591 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f383405cf10 con 0x7f38400b37d0 2024-04-02T16:22:30.004 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:30.004 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f382006c8f0 msgr2=0x7f382006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f382006c8f0 0x7f382006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3838006fd0 tx=0x7f3838008040 comp rx=0 tx=0).stop 2024-04-02T16:22:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38400b37d0 msgr2=0x7f38400c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38400b37d0 0x7f38400c9480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f383400c4a0 tx=0x7f383400c960 comp rx=0 tx=0).stop 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 shutdown_connections 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f384015e430 0x7f384014b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f382006c8f0 0x7f382006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38400b37d0 0x7f38400c9480 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 --2- 172.21.15.50:0/652319591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384006f1f0 0x7f3840162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 >> 172.21.15.50:0/652319591 conn(0x7f38400b5620 msgr2=0x7f38400af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 shutdown_connections 2024-04-02T16:22:30.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.005+0000 7f3844d1c640 1 -- 172.21.15.50:0/652319591 wait complete. 2024-04-02T16:22:30.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:30.017 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:30.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.120+0000 7f613671c640 1 Processor -- start 2024-04-02T16:22:30.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.120+0000 7f613671c640 1 -- start start 2024-04-02T16:22:30.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f613671c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130161ca0 0x7f61300d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f613671c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61300d0c80 0x7f61300d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f613671c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61300d5000 con 0x7f6130162bb0 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f613671c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f61300d1610 con 0x7f61300d0c80 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f613671c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f61300cf0d0 con 0x7f6130161ca0 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f612ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130161ca0 0x7f61300d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f612ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130161ca0 0x7f61300d4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57478/0 (socket says 172.21.15.50:57478) 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f612ffff640 1 -- 172.21.15.50:0/3310558823 learned_addr learned my addr 172.21.15.50:0/3310558823 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f6134cb9640 1 --2- 172.21.15.50:0/3310558823 >> v2:172.21.15.188:3300/0 conn(0x7f61300d0c80 0x7f61300d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f612ffff640 1 -- 172.21.15.50:0/3310558823 >> v2:172.21.15.188:3300/0 conn(0x7f61300d0c80 msgr2=0x7f61300d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f612ffff640 1 --2- 172.21.15.50:0/3310558823 >> v2:172.21.15.188:3300/0 conn(0x7f61300d0c80 0x7f61300d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.122+0000 7f612ffff640 1 -- 172.21.15.50:0/3310558823 >> v1:172.21.15.188:6789/0 conn(0x7f6130162bb0 legacy=0x7f61300cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.121+0000 7f612effd640 1 -- 172.21.15.50:0/3310558823 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3416903816 0 0) 0x7f61300d5000 con 0x7f6130162bb0 2024-04-02T16:22:30.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.122+0000 7f612ffff640 1 -- 172.21.15.50:0/3310558823 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61300cf3b0 con 0x7f6130161ca0 2024-04-02T16:22:30.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.122+0000 7f612ffff640 1 --2- 172.21.15.50:0/3310558823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130161ca0 0x7f61300d4980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6124009870 tx=0x7f612402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919bba6147662596 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.122+0000 7f612effd640 1 -- 172.21.15.50:0/3310558823 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240317c0 con 0x7f6130161ca0 2024-04-02T16:22:30.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.122+0000 7f612effd640 1 -- 172.21.15.50:0/3310558823 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6124037070 con 0x7f6130161ca0 2024-04-02T16:22:30.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.122+0000 7f612effd640 1 -- 172.21.15.50:0/3310558823 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124031b80 con 0x7f6130161ca0 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 -- 172.21.15.50:0/3310558823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130161ca0 msgr2=0x7f61300d4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 --2- 172.21.15.50:0/3310558823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130161ca0 0x7f61300d4980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6124009870 tx=0x7f612402b860 comp rx=0 tx=0).stop 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 -- 172.21.15.50:0/3310558823 shutdown_connections 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 --2- 172.21.15.50:0/3310558823 >> v2:172.21.15.188:3300/0 conn(0x7f61300d0c80 0x7f61300d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 --2- 172.21.15.50:0/3310558823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130161ca0 0x7f61300d4980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 -- 172.21.15.50:0/3310558823 >> 172.21.15.50:0/3310558823 conn(0x7f613009d5c0 msgr2=0x7f61300d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 -- 172.21.15.50:0/3310558823 shutdown_connections 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.123+0000 7f613671c640 1 -- 172.21.15.50:0/3310558823 wait complete. 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.124+0000 7f613671c640 1 Processor -- start 2024-04-02T16:22:30.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.124+0000 7f613671c640 1 -- start start 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f613671c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61300d0c80 0x7f61300ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f613671c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6130161ca0 0x7f61301f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f613671c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130162bb0 0x7f613014dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f613671c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61300ced80 con 0x7f61300d0c80 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f613671c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6130163010 con 0x7f6130161ca0 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f613671c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f61300b70c0 con 0x7f6130162bb0 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f612ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61300d0c80 0x7f61300ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f612ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61300d0c80 0x7f61300ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45438/0 (socket says 172.21.15.50:45438) 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f612ffff640 1 -- 172.21.15.50:0/2346169121 learned_addr learned my addr 172.21.15.50:0/2346169121 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f6134cb9640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130162bb0 0x7f613014dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.125+0000 7f612f7fe640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6130161ca0 0x7f61301f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612ffff640 1 -- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130162bb0 msgr2=0x7f613014dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612ffff640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130162bb0 0x7f613014dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612ffff640 1 -- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6130161ca0 msgr2=0x7f61301f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612ffff640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6130161ca0 0x7f61301f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612ffff640 1 -- 172.21.15.50:0/2346169121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61300c7a40 con 0x7f61300d0c80 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612ffff640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61300d0c80 0x7f61300ce330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6124000c00 tx=0x7f61240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612cff9640 1 -- 172.21.15.50:0/2346169121 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612404a070 con 0x7f61300d0c80 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613014e740 con 0x7f61300d0c80 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612cff9640 1 -- 172.21.15.50:0/2346169121 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6124032450 con 0x7f61300d0c80 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f612cff9640 1 -- 172.21.15.50:0/2346169121 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124032840 con 0x7f61300d0c80 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.126+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613014ed00 con 0x7f61300d0c80 2024-04-02T16:22:30.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.127+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6130075760 con 0x7f61300d0c80 2024-04-02T16:22:30.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.127+0000 7f612cff9640 1 -- 172.21.15.50:0/2346169121 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61240329e0 con 0x7f61300d0c80 2024-04-02T16:22:30.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.128+0000 7f612cff9640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f611c06c8f0 0x7f611c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.128+0000 7f612cff9640 1 -- 172.21.15.50:0/2346169121 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f61240ad840 con 0x7f61300d0c80 2024-04-02T16:22:30.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.128+0000 7f612f7fe640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f611c06c8f0 0x7f611c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.129+0000 7f612f7fe640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f611c06c8f0 0x7f611c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6130160ec0 tx=0x7f6120009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.132+0000 7f612cff9640 1 -- 172.21.15.50:0/2346169121 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6124040070 con 0x7f61300d0c80 2024-04-02T16:22:30.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.315+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61300ba130 con 0x7f61300d0c80 2024-04-02T16:22:30.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.315+0000 7f612cff9640 1 -- 172.21.15.50:0/2346169121 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f612407b260 con 0x7f61300d0c80 2024-04-02T16:22:30.317 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:30.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f611c06c8f0 msgr2=0x7f611c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f611c06c8f0 0x7f611c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6130160ec0 tx=0x7f6120009040 comp rx=0 tx=0).stop 2024-04-02T16:22:30.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61300d0c80 msgr2=0x7f61300ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61300d0c80 0x7f61300ce330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6124000c00 tx=0x7f61240046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 shutdown_connections 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6130162bb0 0x7f613014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6130161ca0 0x7f61301f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f611c06c8f0 0x7f611c06edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 --2- 172.21.15.50:0/2346169121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61300d0c80 0x7f61300ce330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 >> 172.21.15.50:0/2346169121 conn(0x7f613009d5c0 msgr2=0x7f613006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 shutdown_connections 2024-04-02T16:22:30.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.317+0000 7f613671c640 1 -- 172.21.15.50:0/2346169121 wait complete. 2024-04-02T16:22:30.329 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:30.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.432+0000 7f655c463640 1 Processor -- start 2024-04-02T16:22:30.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.433+0000 7f655c463640 1 -- start start 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655c463640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65540adfc0 0x7f655415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655c463640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f65540af240 0x7f655406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655c463640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65540af670 con 0x7f65540d00f0 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655c463640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f65540ae730 con 0x7f65540af240 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655c463640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f655406da50 con 0x7f65540adfc0 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655aa00640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f65540af240 0x7f655406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655aa00640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f65540af240 0x7f655406dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45446/0 (socket says 172.21.15.50:45446) 2024-04-02T16:22:30.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655aa00640 1 -- 172.21.15.50:0/4232877133 learned_addr learned my addr 172.21.15.50:0/4232877133 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655aa00640 1 -- 172.21.15.50:0/4232877133 >> v2:172.21.15.188:3300/0 conn(0x7f65540adfc0 msgr2=0x7f655415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f65599fe640 1 --2- 172.21.15.50:0/4232877133 >> v2:172.21.15.188:3300/0 conn(0x7f65540adfc0 0x7f655415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655aa00640 1 --2- 172.21.15.50:0/4232877133 >> v2:172.21.15.188:3300/0 conn(0x7f65540adfc0 0x7f655415e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655aa00640 1 -- 172.21.15.50:0/4232877133 >> v1:172.21.15.188:6789/0 conn(0x7f65540d00f0 legacy=0x7f65540ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.434+0000 7f655aa00640 1 -- 172.21.15.50:0/4232877133 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65540af940 con 0x7f65540af240 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.435+0000 7f655aa00640 1 --2- 172.21.15.50:0/4232877133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540af240 0x7f655406dee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65540672e0 tx=0x7f654c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de559de714908ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.435+0000 7f65591fd640 1 -- 172.21.15.50:0/4232877133 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c02ca80 con 0x7f65540af240 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.435+0000 7f65591fd640 1 -- 172.21.15.50:0/4232877133 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c02cc20 con 0x7f65540af240 2024-04-02T16:22:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.435+0000 7f65591fd640 1 -- 172.21.15.50:0/4232877133 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0343f0 con 0x7f65540af240 2024-04-02T16:22:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 -- 172.21.15.50:0/4232877133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540af240 msgr2=0x7f655406dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 --2- 172.21.15.50:0/4232877133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540af240 0x7f655406dee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65540672e0 tx=0x7f654c02bee0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 -- 172.21.15.50:0/4232877133 shutdown_connections 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 --2- 172.21.15.50:0/4232877133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540af240 0x7f655406dee0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 --2- 172.21.15.50:0/4232877133 >> v2:172.21.15.188:3300/0 conn(0x7f65540adfc0 0x7f655415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 -- 172.21.15.50:0/4232877133 >> 172.21.15.50:0/4232877133 conn(0x7f6554072c10 msgr2=0x7f655406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 -- 172.21.15.50:0/4232877133 shutdown_connections 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.436+0000 7f655c463640 1 -- 172.21.15.50:0/4232877133 wait complete. 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.437+0000 7f655c463640 1 Processor -- start 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.437+0000 7f655c463640 1 -- start start 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655c463640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540adfc0 0x7f6554155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655c463640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65540af240 0x7f6554154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655c463640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65540d00f0 0x7f655414dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655c463640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f65540b01e0 con 0x7f65540adfc0 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655c463640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65540aff00 con 0x7f65540af240 2024-04-02T16:22:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655c463640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f65540d04d0 con 0x7f65540d00f0 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540adfc0 0x7f6554155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540adfc0 0x7f6554155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45448/0 (socket says 172.21.15.50:45448) 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 -- 172.21.15.50:0/1868379977 learned_addr learned my addr 172.21.15.50:0/1868379977 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655aa00640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65540d00f0 0x7f655414dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f65599fe640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65540af240 0x7f6554154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 -- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65540d00f0 msgr2=0x7f655414dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65540d00f0 0x7f655414dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 -- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65540af240 msgr2=0x7f6554154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65540af240 0x7f6554154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 -- 172.21.15.50:0/1868379977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6554150b00 con 0x7f65540adfc0 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f65599fe640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65540af240 0x7f6554154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.438+0000 7f655a1ff640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540adfc0 0x7f6554155b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f655000c460 tx=0x7f655000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.439+0000 7f654affd640 1 -- 172.21.15.50:0/1868379977 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550016020 con 0x7f65540adfc0 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.439+0000 7f654affd640 1 -- 172.21.15.50:0/1868379977 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6550012870 con 0x7f65540adfc0 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.439+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6554150de0 con 0x7f65540adfc0 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.439+0000 7f654affd640 1 -- 172.21.15.50:0/1868379977 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550012c60 con 0x7f65540adfc0 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.439+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6554156080 con 0x7f65540adfc0 2024-04-02T16:22:30.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.439+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65540b1720 con 0x7f65540adfc0 2024-04-02T16:22:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.440+0000 7f654affd640 1 -- 172.21.15.50:0/1868379977 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6550019020 con 0x7f65540adfc0 2024-04-02T16:22:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.440+0000 7f654affd640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f653806c8f0 0x7f653806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.440+0000 7f654affd640 1 -- 172.21.15.50:0/1868379977 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f655008ba70 con 0x7f65540adfc0 2024-04-02T16:22:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.441+0000 7f65599fe640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f653806c8f0 0x7f653806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.441+0000 7f65599fe640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f653806c8f0 0x7f653806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6544006fd0 tx=0x7f6544008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.445+0000 7f654affd640 1 -- 172.21.15.50:0/1868379977 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f65540b1720 con 0x7f65540adfc0 2024-04-02T16:22:30.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.630+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6554075760 con 0x7f65540adfc0 2024-04-02T16:22:30.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.631+0000 7f654affd640 1 -- 172.21.15.50:0/1868379977 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f6550020030 con 0x7f65540adfc0 2024-04-02T16:22:30.632 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.632+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f653806c8f0 msgr2=0x7f653806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.632+0000 7f655c463640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f653806c8f0 0x7f653806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6544006fd0 tx=0x7f6544008040 comp rx=0 tx=0).stop 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.632+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540adfc0 msgr2=0x7f6554155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.632+0000 7f655c463640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540adfc0 0x7f6554155b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f655000c460 tx=0x7f655000c920 comp rx=0 tx=0).stop 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.632+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 shutdown_connections 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.633+0000 7f655c463640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65540d00f0 0x7f655414dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.633+0000 7f655c463640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65540af240 0x7f6554154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.633+0000 7f655c463640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f653806c8f0 0x7f653806edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.633+0000 7f655c463640 1 --2- 172.21.15.50:0/1868379977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65540adfc0 0x7f6554155b40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.633+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 >> 172.21.15.50:0/1868379977 conn(0x7f6554072c10 msgr2=0x7f6554156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.633+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 shutdown_connections 2024-04-02T16:22:30.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.633+0000 7f655c463640 1 -- 172.21.15.50:0/1868379977 wait complete. 2024-04-02T16:22:30.644 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:30.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.747+0000 7fc3ccdc8640 1 Processor -- start 2024-04-02T16:22:30.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.747+0000 7fc3ccdc8640 1 -- start start 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3ccdc8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc3c8161c80 0x7fc3c80d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3ccdc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d2e50 0x7fc3c80d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3ccdc8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3c80d1200 0x7fc3c80cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3ccdc8640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c80b10e0 con 0x7fc3c8161c80 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3ccdc8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c80b1380 con 0x7fc3c80d1200 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3ccdc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c80b1660 con 0x7fc3c80d2e50 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3c659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc3c8161c80 0x7fc3c80d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3c659c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc3c8161c80 0x7fc3c80d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45460/0 (socket says 172.21.15.50:45460) 2024-04-02T16:22:30.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3c659c640 1 -- 172.21.15.50:0/1121508263 learned_addr learned my addr 172.21.15.50:0/1121508263 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.748+0000 7fc3c5d9b640 1 --2- 172.21.15.50:0/1121508263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d2e50 0x7fc3c80d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.749+0000 7fc3c6d9d640 1 --2- 172.21.15.50:0/1121508263 >> v2:172.21.15.188:3300/0 conn(0x7fc3c80d1200 0x7fc3c80cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.749+0000 7fc3c659c640 1 -- 172.21.15.50:0/1121508263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d2e50 msgr2=0x7fc3c80d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.749+0000 7fc3c659c640 1 --2- 172.21.15.50:0/1121508263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d2e50 0x7fc3c80d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.749+0000 7fc3c659c640 1 -- 172.21.15.50:0/1121508263 >> v2:172.21.15.188:3300/0 conn(0x7fc3c80d1200 msgr2=0x7fc3c80cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.749+0000 7fc3c659c640 1 --2- 172.21.15.50:0/1121508263 >> v2:172.21.15.188:3300/0 conn(0x7fc3c80d1200 0x7fc3c80cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.749+0000 7fc3c659c640 1 -- 172.21.15.50:0/1121508263 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c80d3340 con 0x7fc3c8161c80 2024-04-02T16:22:30.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.749+0000 7fc3c659c640 1 --2- 172.21.15.50:0/1121508263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8161c80 0x7fc3c80d2910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009870 tx=0x7fc3c002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511b95388f5937d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3c559a640 1 -- 172.21.15.50:0/1121508263 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c00317c0 con 0x7fc3c8161c80 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3c559a640 1 -- 172.21.15.50:0/1121508263 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c0037070 con 0x7fc3c8161c80 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3c559a640 1 -- 172.21.15.50:0/1121508263 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0031b80 con 0x7fc3c8161c80 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1121508263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8161c80 msgr2=0x7fc3c80d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1121508263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8161c80 0x7fc3c80d2910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009870 tx=0x7fc3c002b860 comp rx=0 tx=0).stop 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1121508263 shutdown_connections 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1121508263 >> v2:172.21.15.188:3300/0 conn(0x7fc3c80d1200 0x7fc3c80cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1121508263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d2e50 0x7fc3c80d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1121508263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c8161c80 0x7fc3c80d2910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1121508263 >> 172.21.15.50:0/1121508263 conn(0x7fc3c809d5e0 msgr2=0x7fc3c80d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1121508263 shutdown_connections 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.750+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1121508263 wait complete. 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.751+0000 7fc3ccdc8640 1 Processor -- start 2024-04-02T16:22:30.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3ccdc8640 1 -- start start 2024-04-02T16:22:30.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3ccdc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d1200 0x7fc3c815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3ccdc8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c80d2e50 0x7fc3c80cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3ccdc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c8161c80 0x7fc3c80cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3ccdc8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80cf460 con 0x7fc3c80d2e50 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3ccdc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80d0390 con 0x7fc3c8161c80 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3ccdc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c80cf5e0 con 0x7fc3c80d1200 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3c5d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c80d2e50 0x7fc3c80cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3c5d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c80d2e50 0x7fc3c80cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45470/0 (socket says 172.21.15.50:45470) 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3c5d9b640 1 -- 172.21.15.50:0/1878780043 learned_addr learned my addr 172.21.15.50:0/1878780043 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3c659c640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d1200 0x7fc3c815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.752+0000 7fc3c6d9d640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c8161c80 0x7fc3c80cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3c5d9b640 1 -- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d1200 msgr2=0x7fc3c815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3c5d9b640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d1200 0x7fc3c815d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3c5d9b640 1 -- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c8161c80 msgr2=0x7fc3c80cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3c5d9b640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c8161c80 0x7fc3c80cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3c5d9b640 1 -- 172.21.15.50:0/1878780043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c80cc650 con 0x7fc3c80d2e50 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3c5d9b640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c80d2e50 0x7fc3c80cb8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d7d0 tx=0x7fc3b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3beffd640 1 -- 172.21.15.50:0/1878780043 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000b6c0 con 0x7fc3c80d2e50 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3beffd640 1 -- 172.21.15.50:0/1878780043 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b0019070 con 0x7fc3c80d2e50 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c80cc930 con 0x7fc3c80d2e50 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3beffd640 1 -- 172.21.15.50:0/1878780043 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000bac0 con 0x7fc3c80d2e50 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.753+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c80ccef0 con 0x7fc3c80d2e50 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.754+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c8075760 con 0x7fc3c80d2e50 2024-04-02T16:22:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.754+0000 7fc3beffd640 1 -- 172.21.15.50:0/1878780043 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3b000bc60 con 0x7fc3c80d2e50 2024-04-02T16:22:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.755+0000 7fc3beffd640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc3a806c8f0 0x7fc3a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.755+0000 7fc3beffd640 1 -- 172.21.15.50:0/1878780043 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc3b008c1c0 con 0x7fc3c80d2e50 2024-04-02T16:22:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.755+0000 7fc3c659c640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc3a806c8f0 0x7fc3a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.756+0000 7fc3c659c640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc3a806c8f0 0x7fc3a806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0036000 tx=0x7fc3c003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:30.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.759+0000 7fc3beffd640 1 -- 172.21.15.50:0/1878780043 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc3b0059c60 con 0x7fc3c80d2e50 2024-04-02T16:22:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.944+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3c80b9610 con 0x7fc3c80d2e50 2024-04-02T16:22:30.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.944+0000 7fc3beffd640 1 -- 172.21.15.50:0/1878780043 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc3b005df60 con 0x7fc3c80d2e50 2024-04-02T16:22:30.945 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc3a806c8f0 msgr2=0x7fc3a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc3a806c8f0 0x7fc3a806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0036000 tx=0x7fc3c003d040 comp rx=0 tx=0).stop 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c80d2e50 msgr2=0x7fc3c80cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c80d2e50 0x7fc3c80cb8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000d7d0 tx=0x7fc3b000dc90 comp rx=0 tx=0).stop 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 shutdown_connections 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c8161c80 0x7fc3c80cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc3a806c8f0 0x7fc3a806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3c80d2e50 0x7fc3c80cb8e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 --2- 172.21.15.50:0/1878780043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c80d1200 0x7fc3c815d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 >> 172.21.15.50:0/1878780043 conn(0x7fc3c809d5e0 msgr2=0x7fc3c80d35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 shutdown_connections 2024-04-02T16:22:30.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:30.946+0000 7fc3ccdc8640 1 -- 172.21.15.50:0/1878780043 wait complete. 2024-04-02T16:22:30.958 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:31.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.061+0000 7f7808086640 1 Processor -- start 2024-04-02T16:22:31.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.061+0000 7f7808086640 1 -- start start 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7808086640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7800071020 0x7f780009aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7808086640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f780009af80 0x7f78000cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7808086640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f78000cd9d0 0x7f78000cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7808086640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f780009b3c0 con 0x7f78000cd9d0 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7808086640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f780006cc40 con 0x7f780009af80 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7808086640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f780006c960 con 0x7f7800071020 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f78000cd9d0 0x7f78000cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f78000cd9d0 0x7f78000cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45486/0 (socket says 172.21.15.50:45486) 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 -- 172.21.15.50:0/316611034 learned_addr learned my addr 172.21.15.50:0/316611034 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7805621640 1 --2- 172.21.15.50:0/316611034 >> v2:172.21.15.188:3300/0 conn(0x7f780009af80 0x7f78000cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7805e22640 1 --2- 172.21.15.50:0/316611034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7800071020 0x7f780009aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 -- 172.21.15.50:0/316611034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7800071020 msgr2=0x7f780009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:31.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 --2- 172.21.15.50:0/316611034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7800071020 0x7f780009aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 -- 172.21.15.50:0/316611034 >> v2:172.21.15.188:3300/0 conn(0x7f780009af80 msgr2=0x7f78000cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:31.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 --2- 172.21.15.50:0/316611034 >> v2:172.21.15.188:3300/0 conn(0x7f780009af80 0x7f78000cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7806623640 1 -- 172.21.15.50:0/316611034 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78000cc6d0 con 0x7f78000cd9d0 2024-04-02T16:22:31.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.062+0000 7f7805e22640 1 --2- 172.21.15.50:0/316611034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7800071020 0x7f780009aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:31.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.063+0000 7f7806623640 1 --2- 172.21.15.50:0/316611034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78000cd9d0 0x7f78000cde60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7800074c20 tx=0x7f77f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d77b90d684f1eef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.063+0000 7f7804e20640 1 -- 172.21.15.50:0/316611034 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f002cb00 con 0x7f78000cd9d0 2024-04-02T16:22:31.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.063+0000 7f7804e20640 1 -- 172.21.15.50:0/316611034 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f002cca0 con 0x7f78000cd9d0 2024-04-02T16:22:31.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.063+0000 7f7804e20640 1 -- 172.21.15.50:0/316611034 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0034610 con 0x7f78000cd9d0 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 -- 172.21.15.50:0/316611034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78000cd9d0 msgr2=0x7f78000cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 --2- 172.21.15.50:0/316611034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78000cd9d0 0x7f78000cde60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7800074c20 tx=0x7f77f002bf60 comp rx=0 tx=0).stop 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 -- 172.21.15.50:0/316611034 shutdown_connections 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 --2- 172.21.15.50:0/316611034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78000cd9d0 0x7f78000cde60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 --2- 172.21.15.50:0/316611034 >> v2:172.21.15.188:3300/0 conn(0x7f780009af80 0x7f78000cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 --2- 172.21.15.50:0/316611034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7800071020 0x7f780009aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 -- 172.21.15.50:0/316611034 >> 172.21.15.50:0/316611034 conn(0x7f78000b3ad0 msgr2=0x7f78000b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 -- 172.21.15.50:0/316611034 shutdown_connections 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.064+0000 7f7808086640 1 -- 172.21.15.50:0/316611034 wait complete. 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.065+0000 7f7808086640 1 Processor -- start 2024-04-02T16:22:31.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.065+0000 7f7808086640 1 -- start start 2024-04-02T16:22:31.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7808086640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7800071020 0x7f78001545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7808086640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780009af80 0x7f7800160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7808086640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cd9d0 0x7f78000b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7808086640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78000ccf70 con 0x7f780009af80 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7808086640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f78000ccc90 con 0x7f7800071020 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7808086640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f780006db50 con 0x7f78000cd9d0 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780009af80 0x7f7800160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780009af80 0x7f7800160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45492/0 (socket says 172.21.15.50:45492) 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 -- 172.21.15.50:0/3436977364 learned_addr learned my addr 172.21.15.50:0/3436977364 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805e22640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7800071020 0x7f78001545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7806623640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cd9d0 0x7f78000b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 -- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cd9d0 msgr2=0x7f78000b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cd9d0 0x7f78000b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 -- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7800071020 msgr2=0x7f78001545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7800071020 0x7f78001545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 -- 172.21.15.50:0/3436977364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78001618f0 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f7805621640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780009af80 0x7f7800160e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77f400d800 tx=0x7f77f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.066+0000 7f77fe7fc640 1 -- 172.21.15.50:0/3436977364 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f400b6c0 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.067+0000 7f77fe7fc640 1 -- 172.21.15.50:0/3436977364 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f4019070 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.067+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78000b4ec0 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.067+0000 7f77fe7fc640 1 -- 172.21.15.50:0/3436977364 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f400ba80 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.067+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78000b5480 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.067+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7800075760 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.068+0000 7f77fe7fc640 1 -- 172.21.15.50:0/3436977364 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77f400bc20 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.068+0000 7f77fe7fc640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f77ec06c8f0 0x7f77ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.068+0000 7f77fe7fc640 1 -- 172.21.15.50:0/3436977364 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f77f408c360 con 0x7f780009af80 2024-04-02T16:22:31.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.068+0000 7f7805e22640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f77ec06c8f0 0x7f77ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.069+0000 7f7805e22640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f77ec06c8f0 0x7f77ec06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77f800a980 tx=0x7f77f8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.073+0000 7f77fe7fc640 1 -- 172.21.15.50:0/3436977364 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f77f405a450 con 0x7f780009af80 2024-04-02T16:22:31.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.254+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f78000d1ca0 con 0x7f780009af80 2024-04-02T16:22:31.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.254+0000 7f77fe7fc640 1 -- 172.21.15.50:0/3436977364 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f77f4059d50 con 0x7f780009af80 2024-04-02T16:22:31.256 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f77ec06c8f0 msgr2=0x7f77ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f77ec06c8f0 0x7f77ec06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77f800a980 tx=0x7f77f8008040 comp rx=0 tx=0).stop 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780009af80 msgr2=0x7f7800160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780009af80 0x7f7800160e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77f400d800 tx=0x7f77f400dcc0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 shutdown_connections 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cd9d0 0x7f78000b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f77ec06c8f0 0x7f77ec06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f780009af80 0x7f7800160e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 --2- 172.21.15.50:0/3436977364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7800071020 0x7f78001545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 >> 172.21.15.50:0/3436977364 conn(0x7f78000b3ad0 msgr2=0x7f78000b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 shutdown_connections 2024-04-02T16:22:31.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.256+0000 7f7808086640 1 -- 172.21.15.50:0/3436977364 wait complete. 2024-04-02T16:22:31.268 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:22:31.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.368+0000 7fe383338640 1 Processor -- start 2024-04-02T16:22:31.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.369+0000 7fe383338640 1 -- start start 2024-04-02T16:22:31.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.369+0000 7fe383338640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.369+0000 7fe383338640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0d0210 con 0x7fe37c15e670 2024-04-02T16:22:31.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.369+0000 7fe383338640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c06d740 con 0x7fe37c0b4680 2024-04-02T16:22:31.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.369+0000 7fe383338640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c06da20 con 0x7fe37c09b150 2024-04-02T16:22:31.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe3810d4640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe37c15e670 0x7fe37c0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55120/0 (socket says 172.21.15.50:55120) 2024-04-02T16:22:31.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe3810d4640 1 -- 172.21.15.50:0/2570734387 learned_addr learned my addr 172.21.15.50:0/2570734387 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:31.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe3818d5640 1 --2- 172.21.15.50:0/2570734387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe373fff640 1 -- 172.21.15.50:0/2570734387 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194584590 0 0) 0x7fe37c0d0210 con 0x7fe37c15e670 2024-04-02T16:22:31.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe373fff640 1 -- 172.21.15.50:0/2570734387 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe368003520 con 0x7fe37c15e670 2024-04-02T16:22:31.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe3818d5640 1 -- 172.21.15.50:0/2570734387 >> v1:172.21.15.188:6789/0 conn(0x7fe37c0b4680 legacy=0x7fe37c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe3818d5640 1 -- 172.21.15.50:0/2570734387 >> v1:172.21.15.50:6789/0 conn(0x7fe37c15e670 legacy=0x7fe37c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe373fff640 1 -- 172.21.15.50:0/2570734387 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531181571 0 0) 0x7fe37c06d740 con 0x7fe37c0b4680 2024-04-02T16:22:31.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.370+0000 7fe3818d5640 1 -- 172.21.15.50:0/2570734387 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c0d04b0 con 0x7fe37c09b150 2024-04-02T16:22:31.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.371+0000 7fe3818d5640 1 --2- 172.21.15.50:0/2570734387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c06cff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe374009dc0 tx=0x7fe37402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5cf397e185f63b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.371+0000 7fe373fff640 1 -- 172.21.15.50:0/2570734387 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374039070 con 0x7fe37c09b150 2024-04-02T16:22:31.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.371+0000 7fe373fff640 1 -- 172.21.15.50:0/2570734387 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37402cd50 con 0x7fe37c09b150 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.371+0000 7fe373fff640 1 -- 172.21.15.50:0/2570734387 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374034640 con 0x7fe37c09b150 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.371+0000 7fe383338640 1 -- 172.21.15.50:0/2570734387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 msgr2=0x7fe37c06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.372+0000 7fe383338640 1 --2- 172.21.15.50:0/2570734387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c06cff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe374009dc0 tx=0x7fe37402bfa0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.372+0000 7fe383338640 1 -- 172.21.15.50:0/2570734387 shutdown_connections 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.372+0000 7fe383338640 1 --2- 172.21.15.50:0/2570734387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c06cff0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.372+0000 7fe383338640 1 -- 172.21.15.50:0/2570734387 >> 172.21.15.50:0/2570734387 conn(0x7fe37c0b3b90 msgr2=0x7fe37c0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.372+0000 7fe383338640 1 -- 172.21.15.50:0/2570734387 shutdown_connections 2024-04-02T16:22:31.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.372+0000 7fe383338640 1 -- 172.21.15.50:0/2570734387 wait complete. 2024-04-02T16:22:31.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.373+0000 7fe383338640 1 Processor -- start 2024-04-02T16:22:31.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.373+0000 7fe383338640 1 -- start start 2024-04-02T16:22:31.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe383338640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c1f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe383338640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c0b4680 0x7fe37c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe383338640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c15e670 0x7fe37c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe383338640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0d16a0 con 0x7fe37c15e670 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe383338640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0d0b00 con 0x7fe37c0b4680 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe383338640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c06f5a0 con 0x7fe37c09b150 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c15e670 0x7fe37c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c15e670 0x7fe37c0b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45508/0 (socket says 172.21.15.50:45508) 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 -- 172.21.15.50:0/2761348049 learned_addr learned my addr 172.21.15.50:0/2761348049 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3810d4640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c1f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3808d3640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c0b4680 0x7fe37c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 -- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 msgr2=0x7fe37c1f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c1f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 -- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c0b4680 msgr2=0x7fe37c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c0b4680 0x7fe37c0b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 -- 172.21.15.50:0/2761348049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1615b0 con 0x7fe37c15e670 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3810d4640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c1f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3808d3640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c0b4680 0x7fe37c0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:31.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe3818d5640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c15e670 0x7fe37c0b30d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c5b0 tx=0x7fe374004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe371ffb640 1 -- 172.21.15.50:0/2761348049 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374042020 con 0x7fe37c15e670 2024-04-02T16:22:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe371ffb640 1 -- 172.21.15.50:0/2761348049 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37403a040 con 0x7fe37c15e670 2024-04-02T16:22:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1f7380 con 0x7fe37c15e670 2024-04-02T16:22:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.374+0000 7fe371ffb640 1 -- 172.21.15.50:0/2761348049 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374039070 con 0x7fe37c15e670 2024-04-02T16:22:31.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.375+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c161890 con 0x7fe37c15e670 2024-04-02T16:22:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.376+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c0756f0 con 0x7fe37c15e670 2024-04-02T16:22:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.376+0000 7fe371ffb640 1 -- 172.21.15.50:0/2761348049 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe374045020 con 0x7fe37c15e670 2024-04-02T16:22:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.377+0000 7fe371ffb640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe36006c920 0x7fe36006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.377+0000 7fe371ffb640 1 -- 172.21.15.50:0/2761348049 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe3740ae570 con 0x7fe37c15e670 2024-04-02T16:22:31.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.377+0000 7fe3810d4640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe36006c920 0x7fe36006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.377+0000 7fe3810d4640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe36006c920 0x7fe36006ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe378007fd0 tx=0x7fe378009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.381+0000 7fe371ffb640 1 -- 172.21.15.50:0/2761348049 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe37404c020 con 0x7fe37c15e670 2024-04-02T16:22:31.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.608+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe37c0b8020 con 0x7fe37c15e670 2024-04-02T16:22:31.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.608+0000 7fe371ffb640 1 -- 172.21.15.50:0/2761348049 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7fe37407c780 con 0x7fe37c15e670 2024-04-02T16:22:31.610 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:31.610 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:22:31.610 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.610+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe36006c920 msgr2=0x7fe36006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.610+0000 7fe383338640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe36006c920 0x7fe36006ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe378007fd0 tx=0x7fe378009040 comp rx=0 tx=0).stop 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.610+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c15e670 msgr2=0x7fe37c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.610+0000 7fe383338640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c15e670 0x7fe37c0b30d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c5b0 tx=0x7fe374004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 shutdown_connections 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe36006c920 0x7fe36006ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c15e670 0x7fe37c0b30d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe37c0b4680 0x7fe37c0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 --2- 172.21.15.50:0/2761348049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe37c09b150 0x7fe37c1f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 >> 172.21.15.50:0/2761348049 conn(0x7fe37c0b3b90 msgr2=0x7fe37c0b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:31.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 shutdown_connections 2024-04-02T16:22:31.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.611+0000 7fe383338640 1 -- 172.21.15.50:0/2761348049 wait complete. 2024-04-02T16:22:31.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:31.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.723+0000 7fa6f23e9640 1 Processor -- start 2024-04-02T16:22:31.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.723+0000 7fa6f23e9640 1 -- start start 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f23e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f23e9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa6ec06cff0 0x7fa6ec0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f23e9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6ec09aa40 0x7fa6ec09aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f23e9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ec06da20 con 0x7fa6ec06cff0 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f23e9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ec0d0b00 con 0x7fa6ec09aa40 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f23e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec0d0210 con 0x7fa6ec15e670 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa6ec06cff0 0x7fa6ec0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa6ec06cff0 0x7fa6ec0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45512/0 (socket says 172.21.15.50:45512) 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 -- 172.21.15.50:0/3072312393 learned_addr learned my addr 172.21.15.50:0/3072312393 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6ebfff640 1 --2- 172.21.15.50:0/3072312393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 -- 172.21.15.50:0/3072312393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 msgr2=0x7fa6ec0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:31.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 --2- 172.21.15.50:0/3072312393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec0b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 -- 172.21.15.50:0/3072312393 >> v2:172.21.15.188:3300/0 conn(0x7fa6ec09aa40 msgr2=0x7fa6ec09aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f0986640 1 --2- 172.21.15.50:0/3072312393 >> v2:172.21.15.188:3300/0 conn(0x7fa6ec09aa40 0x7fa6ec09aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 --2- 172.21.15.50:0/3072312393 >> v2:172.21.15.188:3300/0 conn(0x7fa6ec09aa40 0x7fa6ec09aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6eb7fe640 1 -- 172.21.15.50:0/3072312393 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec06d740 con 0x7fa6ec06cff0 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6ebfff640 1 --2- 172.21.15.50:0/3072312393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.724+0000 7fa6f0986640 1 --2- 172.21.15.50:0/3072312393 >> v2:172.21.15.188:3300/0 conn(0x7fa6ec09aa40 0x7fa6ec09aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.725+0000 7fa6eb7fe640 1 --2- 172.21.15.50:0/3072312393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec0d0f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec073160 tx=0x7fa6dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee510bd45cb9676 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.725+0000 7fa6eaffd640 1 -- 172.21.15.50:0/3072312393 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc034070 con 0x7fa6ec06cff0 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.725+0000 7fa6eaffd640 1 -- 172.21.15.50:0/3072312393 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6dc030860 con 0x7fa6ec06cff0 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.725+0000 7fa6eaffd640 1 -- 172.21.15.50:0/3072312393 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc030c30 con 0x7fa6ec06cff0 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 -- 172.21.15.50:0/3072312393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 msgr2=0x7fa6ec0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/3072312393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec0d0f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec073160 tx=0x7fa6dc02b9a0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 -- 172.21.15.50:0/3072312393 shutdown_connections 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/3072312393 >> v2:172.21.15.188:3300/0 conn(0x7fa6ec09aa40 0x7fa6ec09aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/3072312393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec0d0f90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/3072312393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec0b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 -- 172.21.15.50:0/3072312393 >> 172.21.15.50:0/3072312393 conn(0x7fa6ec0b3b90 msgr2=0x7fa6ec0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:31.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 -- 172.21.15.50:0/3072312393 shutdown_connections 2024-04-02T16:22:31.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.726+0000 7fa6f23e9640 1 -- 172.21.15.50:0/3072312393 wait complete. 2024-04-02T16:22:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.727+0000 7fa6f23e9640 1 Processor -- start 2024-04-02T16:22:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.727+0000 7fa6f23e9640 1 -- start start 2024-04-02T16:22:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f23e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f23e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ec09aa40 0x7fa6ec0b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f23e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f23e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec06d450 con 0x7fa6ec06cff0 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f23e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec0d0790 con 0x7fa6ec09aa40 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f23e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec0d14d0 con 0x7fa6ec15e670 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45518/0 (socket says 172.21.15.50:45518) 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 -- 172.21.15.50:0/2759301754 learned_addr learned my addr 172.21.15.50:0/2759301754 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6eb7fe640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ec09aa40 0x7fa6ec0b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f0986640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 -- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 msgr2=0x7fa6ec15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 -- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ec09aa40 msgr2=0x7fa6ec0b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ec09aa40 0x7fa6ec0b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 -- 172.21.15.50:0/2759301754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec1f7610 con 0x7fa6ec06cff0 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f0986640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6ebfff640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec159690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c7f0 tx=0x7fa6e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6e8ff9640 1 -- 172.21.15.50:0/2759301754 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0015070 con 0x7fa6ec06cff0 2024-04-02T16:22:31.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6e8ff9640 1 -- 172.21.15.50:0/2759301754 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e0007820 con 0x7fa6ec06cff0 2024-04-02T16:22:31.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec1f78f0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.728+0000 7fa6e8ff9640 1 -- 172.21.15.50:0/2759301754 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0007bd0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.729+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec1f7eb0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.730+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec0745a0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.730+0000 7fa6e8ff9640 1 -- 172.21.15.50:0/2759301754 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e0018070 con 0x7fa6ec06cff0 2024-04-02T16:22:31.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.731+0000 7fa6e8ff9640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6d806c8f0 0x7fa6d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:31.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.731+0000 7fa6e8ff9640 1 -- 172.21.15.50:0/2759301754 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa6e008bdc0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.731+0000 7fa6eb7fe640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6d806c8f0 0x7fa6d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:31.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.731+0000 7fa6eb7fe640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6d806c8f0 0x7fa6d806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc036000 tx=0x7fa6dc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:31.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.735+0000 7fa6e8ff9640 1 -- 172.21.15.50:0/2759301754 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa6ec0745a0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.920+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6ec0756f0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.921+0000 7fa6e8ff9640 1 -- 172.21.15.50:0/2759301754 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fa6e00597c0 con 0x7fa6ec06cff0 2024-04-02T16:22:31.922 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:31.923 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6d806c8f0 msgr2=0x7fa6d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6d806c8f0 0x7fa6d806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc036000 tx=0x7fa6dc008030 comp rx=0 tx=0).stop 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 msgr2=0x7fa6ec159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec159690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c7f0 tx=0x7fa6e000ccb0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 shutdown_connections 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ec15e670 0x7fa6ec15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ec09aa40 0x7fa6ec0b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6d806c8f0 0x7fa6d806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 --2- 172.21.15.50:0/2759301754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6ec06cff0 0x7fa6ec159690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 >> 172.21.15.50:0/2759301754 conn(0x7fa6ec0b3b90 msgr2=0x7fa6ec162980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 shutdown_connections 2024-04-02T16:22:31.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:31.923+0000 7fa6f23e9640 1 -- 172.21.15.50:0/2759301754 wait complete. 2024-04-02T16:22:31.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:31.936 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:32.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.039+0000 7f26ec01b640 1 Processor -- start 2024-04-02T16:22:32.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.039+0000 7f26ec01b640 1 -- start start 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26ec01b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f26e40cf910 0x7f26e40d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26ec01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 0x7f26e40b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26ec01b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e415eae0 con 0x7f26e415e700 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26ec01b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f26e40b4310 con 0x7f26e40cf910 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26ec01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26e40cfe60 con 0x7f26e40d2e10 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26e9db7640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f26e415e700 0x7f26e40d20d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55126/0 (socket says 172.21.15.50:55126) 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26e9db7640 1 -- 172.21.15.50:0/3035478246 learned_addr learned my addr 172.21.15.50:0/3035478246 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26e95b6640 1 --2- 172.21.15.50:0/3035478246 >> v2:172.21.15.188:3300/0 conn(0x7f26e40cf910 0x7f26e40d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26ea5b8640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 0x7f26e40b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479178360 0 0) 0x7f26e415eae0 con 0x7f26e415e700 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.040+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26cc003520 con 0x7f26e415e700 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2908283002 0 0) 0x7f26cc003520 con 0x7f26e415e700 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 msgr2=0x7f26e40b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 0x7f26e40b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 >> v2:172.21.15.188:3300/0 conn(0x7f26e40cf910 msgr2=0x7f26e40d28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 --2- 172.21.15.50:0/3035478246 >> v2:172.21.15.188:3300/0 conn(0x7f26e40cf910 0x7f26e40d28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e40d0140 con 0x7f26e415e700 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f26dc001670 con 0x7f26e415e700 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 >> v1:172.21.15.50:6789/0 conn(0x7f26e415e700 legacy=0x7f26e40d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e95b6640 1 --2- 172.21.15.50:0/3035478246 >> v2:172.21.15.188:3300/0 conn(0x7f26e40cf910 0x7f26e40d28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 0x7f26cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26cc003f30 0x7f26cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc004d40 0x7f26cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26e40b4310 con 0x7f26cc003f30 2024-04-02T16:22:32.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26e40cfe60 con 0x7f26cc004d40 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26cc00b9c0 con 0x7f26e40d2e10 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e95b6640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26cc003f30 0x7f26cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e9db7640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 0x7f26cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26ea5b8640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc004d40 0x7f26cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e95b6640 1 -- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 msgr2=0x7f26cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e95b6640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40d2e10 0x7f26cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e95b6640 1 -- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc004d40 msgr2=0x7f26cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e95b6640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc004d40 0x7f26cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.041+0000 7f26e95b6640 1 -- 172.21.15.50:0/3035478246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc003520 con 0x7f26cc003f30 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26e95b6640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26cc003f30 0x7f26cc004800 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009c90 tx=0x7f26cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b787b1637d0863f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8039070 con 0x7f26cc003f30 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d802c980 con 0x7f26cc003f30 2024-04-02T16:22:32.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26e8db5640 1 -- 172.21.15.50:0/3035478246 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8009860 con 0x7f26cc003f30 2024-04-02T16:22:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26ec01b640 1 -- 172.21.15.50:0/3035478246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26cc003f30 msgr2=0x7f26cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26ec01b640 1 --2- 172.21.15.50:0/3035478246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26cc003f30 0x7f26cc004800 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009c90 tx=0x7f26cc005830 comp rx=0 tx=0).stop 2024-04-02T16:22:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26ea5b8640 1 -- 172.21.15.50:0/3035478246 reap_dead start 2024-04-02T16:22:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.042+0000 7f26ec01b640 1 -- 172.21.15.50:0/3035478246 shutdown_connections 2024-04-02T16:22:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.043+0000 7f26ec01b640 1 -- 172.21.15.50:0/3035478246 >> 172.21.15.50:0/3035478246 conn(0x7f26e409d5c0 msgr2=0x7f26e40d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.043+0000 7f26ec01b640 1 -- 172.21.15.50:0/3035478246 shutdown_connections 2024-04-02T16:22:32.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.043+0000 7f26ec01b640 1 -- 172.21.15.50:0/3035478246 wait complete. 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 Processor -- start 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 -- start start 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc003f30 0x7f26e4152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4151f50 0x7f26e4151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40b2ca0 0x7f26e415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26e40b47a0 con 0x7f26e4151f50 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26e40b3fa0 con 0x7f26cc003f30 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.044+0000 7f26ec01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26e40b4920 con 0x7f26e40b2ca0 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4151f50 0x7f26e4151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4151f50 0x7f26e4151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45536/0 (socket says 172.21.15.50:45536) 2024-04-02T16:22:32.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 -- 172.21.15.50:0/2750115944 learned_addr learned my addr 172.21.15.50:0/2750115944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e9db7640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc003f30 0x7f26e4152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26ea5b8640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40b2ca0 0x7f26e415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 -- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40b2ca0 msgr2=0x7f26e415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40b2ca0 0x7f26e415ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 -- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc003f30 msgr2=0x7f26e4152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc003f30 0x7f26e4152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 -- 172.21.15.50:0/2750115944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e40b1f00 con 0x7f26e4151f50 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26ea5b8640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40b2ca0 0x7f26e415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e95b6640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4151f50 0x7f26e4151860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26d802c590 tx=0x7f26d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e27fc640 1 -- 172.21.15.50:0/2750115944 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8039050 con 0x7f26e4151f50 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e27fc640 1 -- 172.21.15.50:0/2750115944 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d803a040 con 0x7f26e4151f50 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26e27fc640 1 -- 172.21.15.50:0/2750115944 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8009860 con 0x7f26e4151f50 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e4161720 con 0x7f26e4151f50 2024-04-02T16:22:32.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.045+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e40b21c0 con 0x7f26e4151f50 2024-04-02T16:22:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.046+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e4075760 con 0x7f26e4151f50 2024-04-02T16:22:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.047+0000 7f26e27fc640 1 -- 172.21.15.50:0/2750115944 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26d8047020 con 0x7f26e4151f50 2024-04-02T16:22:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.047+0000 7f26e27fc640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f26d006c8f0 0x7f26d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.047+0000 7f26e27fc640 1 -- 172.21.15.50:0/2750115944 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f26d80ada00 con 0x7f26e4151f50 2024-04-02T16:22:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.047+0000 7f26e9db7640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f26d006c8f0 0x7f26d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.048+0000 7f26e9db7640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f26d006c8f0 0x7f26d006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26dc007fd0 tx=0x7f26dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.051+0000 7f26e27fc640 1 -- 172.21.15.50:0/2750115944 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f26d807baf0 con 0x7f26e4151f50 2024-04-02T16:22:32.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.238+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26e40d0730 con 0x7f26e4151f50 2024-04-02T16:22:32.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.239+0000 7f26e27fc640 1 -- 172.21.15.50:0/2750115944 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f26d807b3f0 con 0x7f26e4151f50 2024-04-02T16:22:32.240 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f26d006c8f0 msgr2=0x7f26d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f26d006c8f0 0x7f26d006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26dc007fd0 tx=0x7f26dc009040 comp rx=0 tx=0).stop 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4151f50 msgr2=0x7f26e4151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4151f50 0x7f26e4151860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26d802c590 tx=0x7f26d8004730 comp rx=0 tx=0).stop 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 shutdown_connections 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26e40b2ca0 0x7f26e415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f26d006c8f0 0x7f26d006edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4151f50 0x7f26e4151860 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 --2- 172.21.15.50:0/2750115944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26cc003f30 0x7f26e4152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 >> 172.21.15.50:0/2750115944 conn(0x7f26e409d5c0 msgr2=0x7f26e40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 shutdown_connections 2024-04-02T16:22:32.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.241+0000 7f26ec01b640 1 -- 172.21.15.50:0/2750115944 wait complete. 2024-04-02T16:22:32.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:32.310 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:22:32.313 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:22:32.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.357+0000 7fcadd511640 1 Processor -- start 2024-04-02T16:22:32.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.357+0000 7fcadd511640 1 -- start start 2024-04-02T16:22:32.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcadd511640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcad8162600 0x7fcad80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcadd511640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcad8162f30 0x7fcad80d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcadd511640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad809b450 con 0x7fcad80d1470 2024-04-02T16:22:32.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcadd511640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fcad80d0aa0 con 0x7fcad8162600 2024-04-02T16:22:32.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcadd511640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcad809b6f0 con 0x7fcad8162f30 2024-04-02T16:22:32.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcad6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcad8162600 0x7fcad80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcad6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fcad8162600 0x7fcad80b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45544/0 (socket says 172.21.15.50:45544) 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcad6ffd640 1 -- 172.21.15.50:0/132721260 learned_addr learned my addr 172.21.15.50:0/132721260 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.358+0000 7fcad67fc640 1 --2- 172.21.15.50:0/132721260 >> v2:172.21.15.188:3300/0 conn(0x7fcad8162f30 0x7fcad80d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.359+0000 7fcad6ffd640 1 -- 172.21.15.50:0/132721260 >> v2:172.21.15.188:3300/0 conn(0x7fcad8162f30 msgr2=0x7fcad80d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.359+0000 7fcad6ffd640 1 --2- 172.21.15.50:0/132721260 >> v2:172.21.15.188:3300/0 conn(0x7fcad8162f30 0x7fcad80d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.359+0000 7fcad6ffd640 1 -- 172.21.15.50:0/132721260 >> v1:172.21.15.188:6789/0 conn(0x7fcad80d1470 legacy=0x7fcad80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.359+0000 7fcad6ffd640 1 -- 172.21.15.50:0/132721260 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad809b9d0 con 0x7fcad8162600 2024-04-02T16:22:32.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.359+0000 7fcad6ffd640 1 --2- 172.21.15.50:0/132721260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162600 0x7fcad80b4740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009e50 tx=0x7fcacc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d3d12e7f468f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcad5ffb640 1 -- 172.21.15.50:0/132721260 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc02cb20 con 0x7fcad8162600 2024-04-02T16:22:32.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcad5ffb640 1 -- 172.21.15.50:0/132721260 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcacc02ccc0 con 0x7fcad8162600 2024-04-02T16:22:32.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcad5ffb640 1 -- 172.21.15.50:0/132721260 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc034610 con 0x7fcad8162600 2024-04-02T16:22:32.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 -- 172.21.15.50:0/132721260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162600 msgr2=0x7fcad80b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 --2- 172.21.15.50:0/132721260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162600 0x7fcad80b4740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009e50 tx=0x7fcacc02bf80 comp rx=0 tx=0).stop 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 -- 172.21.15.50:0/132721260 shutdown_connections 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 --2- 172.21.15.50:0/132721260 >> v2:172.21.15.188:3300/0 conn(0x7fcad8162f30 0x7fcad80d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 --2- 172.21.15.50:0/132721260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162600 0x7fcad80b4740 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 -- 172.21.15.50:0/132721260 >> 172.21.15.50:0/132721260 conn(0x7fcad80b3c20 msgr2=0x7fcad80b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 -- 172.21.15.50:0/132721260 shutdown_connections 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.360+0000 7fcadd511640 1 -- 172.21.15.50:0/132721260 wait complete. 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.361+0000 7fcadd511640 1 Processor -- start 2024-04-02T16:22:32.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.361+0000 7fcadd511640 1 -- start start 2024-04-02T16:22:32.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcadd511640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcad80d1470 0x7fcad81f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcadd511640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcad8162600 0x7fcad81f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcadd511640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162f30 0x7fcad80c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcadd511640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80d04d0 con 0x7fcad8162f30 2024-04-02T16:22:32.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcadd511640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80d0350 con 0x7fcad80d1470 2024-04-02T16:22:32.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcadd511640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcad80cce90 con 0x7fcad8162600 2024-04-02T16:22:32.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcad77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162f30 0x7fcad80c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcad77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162f30 0x7fcad80c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45552/0 (socket says 172.21.15.50:45552) 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcad77fe640 1 -- 172.21.15.50:0/3439327333 learned_addr learned my addr 172.21.15.50:0/3439327333 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcad6ffd640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcad80d1470 0x7fcad81f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.362+0000 7fcad67fc640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcad8162600 0x7fcad81f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcad77fe640 1 -- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcad8162600 msgr2=0x7fcad81f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcad77fe640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcad8162600 0x7fcad81f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcad77fe640 1 -- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcad80d1470 msgr2=0x7fcad81f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcad77fe640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcad80d1470 0x7fcad81f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcad77fe640 1 -- 172.21.15.50:0/3439327333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad80c6dc0 con 0x7fcad8162f30 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcad77fe640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162f30 0x7fcad80c7e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c3c0 tx=0x7fcad000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcabf7fe640 1 -- 172.21.15.50:0/3439327333 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0017070 con 0x7fcad8162f30 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcabf7fe640 1 -- 172.21.15.50:0/3439327333 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad00052d0 con 0x7fcad8162f30 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad80c70a0 con 0x7fcad8162f30 2024-04-02T16:22:32.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcabf7fe640 1 -- 172.21.15.50:0/3439327333 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0003850 con 0x7fcad8162f30 2024-04-02T16:22:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.363+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad80c7660 con 0x7fcad8162f30 2024-04-02T16:22:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.364+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad8070d40 con 0x7fcad8162f30 2024-04-02T16:22:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.364+0000 7fcabf7fe640 1 -- 172.21.15.50:0/3439327333 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcad001a070 con 0x7fcad8162f30 2024-04-02T16:22:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.365+0000 7fcabf7fe640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.365+0000 7fcabf7fe640 1 -- 172.21.15.50:0/3439327333 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcad008b090 con 0x7fcad8162f30 2024-04-02T16:22:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.365+0000 7fcad6ffd640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.366+0000 7fcad6ffd640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcac406c8f0 0x7fcac406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcacc035000 tx=0x7fcacc030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.369+0000 7fcabf7fe640 1 -- 172.21.15.50:0/3439327333 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcad8070d40 con 0x7fcad8162f30 2024-04-02T16:22:32.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.554+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcad80b80b0 con 0x7fcad8162f30 2024-04-02T16:22:32.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.554+0000 7fcabf7fe640 1 -- 172.21.15.50:0/3439327333 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fcad0058bc0 con 0x7fcad8162f30 2024-04-02T16:22:32.555 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:32.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcac406c8f0 msgr2=0x7fcac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcac406c8f0 0x7fcac406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcacc035000 tx=0x7fcacc030000 comp rx=0 tx=0).stop 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162f30 msgr2=0x7fcad80c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162f30 0x7fcad80c7e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c3c0 tx=0x7fcad000c880 comp rx=0 tx=0).stop 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 shutdown_connections 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcad8162f30 0x7fcad80c7e60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcad8162600 0x7fcad81f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 --2- 172.21.15.50:0/3439327333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcad80d1470 0x7fcad81f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 >> 172.21.15.50:0/3439327333 conn(0x7fcad80b3c20 msgr2=0x7fcad815d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 shutdown_connections 2024-04-02T16:22:32.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.556+0000 7fcadd511640 1 -- 172.21.15.50:0/3439327333 wait complete. 2024-04-02T16:22:32.568 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:32.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.717+0000 7fc1f311c640 1 Processor -- start 2024-04-02T16:22:32.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.717+0000 7fc1f311c640 1 -- start start 2024-04-02T16:22:32.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.718+0000 7fc1f311c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec0d4170 0x7fc1ec0d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.718+0000 7fc1f311c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1ec0d4b10 con 0x7fc1ec0d0850 2024-04-02T16:22:32.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.718+0000 7fc1f311c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1ec0cf7b0 con 0x7fc1ec15e5d0 2024-04-02T16:22:32.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.718+0000 7fc1f311c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec15e9b0 con 0x7fc1ec0d4170 2024-04-02T16:22:32.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.718+0000 7fc1f16b9640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc1ec0d0850 0x7fc1ec0d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55142/0 (socket says 172.21.15.50:55142) 2024-04-02T16:22:32.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.718+0000 7fc1f16b9640 1 -- 172.21.15.50:0/4151297927 learned_addr learned my addr 172.21.15.50:0/4151297927 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:32.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.718+0000 7fc1ebfff640 1 --2- 172.21.15.50:0/4151297927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec0d4170 0x7fc1ec0d45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984241797 0 0) 0x7fc1ec0d4b10 con 0x7fc1ec0d0850 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1d4003520 con 0x7fc1ec0d0850 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 613870870 0 0) 0x7fc1ec0cf7b0 con 0x7fc1ec15e5d0 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1ec0d4b10 con 0x7fc1ec15e5d0 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3026754435 0 0) 0x7fc1d4003520 con 0x7fc1ec0d0850 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1ebfff640 1 -- 172.21.15.50:0/4151297927 >> v1:172.21.15.188:6789/0 conn(0x7fc1ec15e5d0 legacy=0x7fc1ec0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1ebfff640 1 -- 172.21.15.50:0/4151297927 >> v1:172.21.15.50:6789/0 conn(0x7fc1ec0d0850 legacy=0x7fc1ec0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.719+0000 7fc1ebfff640 1 -- 172.21.15.50:0/4151297927 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec0cfa90 con 0x7fc1ec0d4170 2024-04-02T16:22:32.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1ebfff640 1 --2- 172.21.15.50:0/4151297927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec0d4170 0x7fc1ec0d45d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00097d0 tx=0x7fc1e002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d50b69cdec746a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0037070 con 0x7fc1ec0d4170 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e0030510 con 0x7fc1ec0d4170 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1eb7fe640 1 -- 172.21.15.50:0/4151297927 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc001670 con 0x7fc1ec0d4170 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1f311c640 1 -- 172.21.15.50:0/4151297927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec0d4170 msgr2=0x7fc1ec0d45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1f311c640 1 --2- 172.21.15.50:0/4151297927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec0d4170 0x7fc1ec0d45d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00097d0 tx=0x7fc1e002b800 comp rx=0 tx=0).stop 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1f311c640 1 -- 172.21.15.50:0/4151297927 shutdown_connections 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1f311c640 1 --2- 172.21.15.50:0/4151297927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec0d4170 0x7fc1ec0d45d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1f311c640 1 -- 172.21.15.50:0/4151297927 >> 172.21.15.50:0/4151297927 conn(0x7fc1ec08dce0 msgr2=0x7fc1ec09d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1f311c640 1 -- 172.21.15.50:0/4151297927 shutdown_connections 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.720+0000 7fc1f311c640 1 -- 172.21.15.50:0/4151297927 wait complete. 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 Processor -- start 2024-04-02T16:22:32.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 -- start start 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1ec0d0850 0x7fc1ec1f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec0d4170 0x7fc1ec1f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e5d0 0x7fc1ec0b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0d0050 con 0x7fc1ec0d0850 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0d03c0 con 0x7fc1ec0d4170 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.722+0000 7fc1f311c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0cf3e0 con 0x7fc1ec15e5d0 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1ec0d0850 0x7fc1ec1f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1ec0d0850 0x7fc1ec1f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45562/0 (socket says 172.21.15.50:45562) 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 -- 172.21.15.50:0/3416957140 learned_addr learned my addr 172.21.15.50:0/3416957140 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1ebfff640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec0d4170 0x7fc1ec1f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f16b9640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e5d0 0x7fc1ec0b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 -- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e5d0 msgr2=0x7fc1ec0b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e5d0 0x7fc1ec0b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 -- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec0d4170 msgr2=0x7fc1ec1f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec0d4170 0x7fc1ec1f83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 -- 172.21.15.50:0/3416957140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec0b8850 con 0x7fc1ec0d0850 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f0eb8640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1ec0d0850 0x7fc1ec1f7e90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c7a0 tx=0x7fc1e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1e97fa640 1 -- 172.21.15.50:0/3416957140 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4003b70 con 0x7fc1ec0d0850 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1e97fa640 1 -- 172.21.15.50:0/3416957140 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e4003d10 con 0x7fc1ec0d0850 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec0b8b30 con 0x7fc1ec0d0850 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1e97fa640 1 -- 172.21.15.50:0/3416957140 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e400aa90 con 0x7fc1ec0d0850 2024-04-02T16:22:32.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.723+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec14a570 con 0x7fc1ec0d0850 2024-04-02T16:22:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.724+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ec075760 con 0x7fc1ec0d0850 2024-04-02T16:22:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.725+0000 7fc1e97fa640 1 -- 172.21.15.50:0/3416957140 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1e400ac30 con 0x7fc1ec0d0850 2024-04-02T16:22:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.725+0000 7fc1e97fa640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc1d806c8f0 0x7fc1d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.725+0000 7fc1e97fa640 1 -- 172.21.15.50:0/3416957140 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc1e408bcc0 con 0x7fc1ec0d0850 2024-04-02T16:22:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.725+0000 7fc1ebfff640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc1d806c8f0 0x7fc1d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:32.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.726+0000 7fc1ebfff640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc1d806c8f0 0x7fc1d806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0034000 tx=0x7fc1e0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:32.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.729+0000 7fc1e97fa640 1 -- 172.21.15.50:0/3416957140 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1e4059760 con 0x7fc1ec0d0850 2024-04-02T16:22:32.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.915+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1ec0b5730 con 0x7fc1ec0d0850 2024-04-02T16:22:32.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.915+0000 7fc1e97fa640 1 -- 172.21.15.50:0/3416957140 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc1e405da60 con 0x7fc1ec0d0850 2024-04-02T16:22:32.917 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:32.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc1d806c8f0 msgr2=0x7fc1d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc1d806c8f0 0x7fc1d806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0034000 tx=0x7fc1e0008030 comp rx=0 tx=0).stop 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1ec0d0850 msgr2=0x7fc1ec1f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1ec0d0850 0x7fc1ec1f7e90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c7a0 tx=0x7fc1e400cc60 comp rx=0 tx=0).stop 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 shutdown_connections 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e5d0 0x7fc1ec0b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec0d4170 0x7fc1ec1f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc1d806c8f0 0x7fc1d806edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 --2- 172.21.15.50:0/3416957140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1ec0d0850 0x7fc1ec1f7e90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 >> 172.21.15.50:0/3416957140 conn(0x7fc1ec08dce0 msgr2=0x7fc1ec0d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 shutdown_connections 2024-04-02T16:22:32.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:32.917+0000 7fc1f311c640 1 -- 172.21.15.50:0/3416957140 wait complete. 2024-04-02T16:22:32.929 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:33.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7faff171c640 1 Processor -- start 2024-04-02T16:22:33.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7faff171c640 1 -- start start 2024-04-02T16:22:33.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7faff171c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fafec15e3d0 0x7fafec0d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7faff171c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e7b0 0x7fafec0d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7faff171c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafec0d2f30 con 0x7fafec0c7e30 2024-04-02T16:22:33.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7faff171c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fafec0cf000 con 0x7fafec15e3d0 2024-04-02T16:22:33.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7faff171c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafec0cf2e0 con 0x7fafec15e7b0 2024-04-02T16:22:33.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeb7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e7b0 0x7fafec0d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeb7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e7b0 0x7fafec0d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:57624/0 (socket says 172.21.15.50:57624) 2024-04-02T16:22:33.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeaffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fafec15e3d0 0x7fafec0d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeb7fe640 1 -- 172.21.15.50:0/1051621132 learned_addr learned my addr 172.21.15.50:0/1051621132 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeaffd640 1 -- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e7b0 msgr2=0x7fafec0d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeaffd640 1 --2- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e7b0 0x7fafec0d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeaffd640 1 -- 172.21.15.50:0/1051621132 >> v1:172.21.15.188:6789/0 conn(0x7fafec0c7e30 legacy=0x7fafec0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafea7fc640 1 -- 172.21.15.50:0/1051621132 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 378268070 0 0) 0x7fafec0d2f30 con 0x7fafec0c7e30 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeaffd640 1 -- 172.21.15.50:0/1051621132 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec0cf5c0 con 0x7fafec15e3d0 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.034+0000 7fafeb7fe640 1 --2- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e7b0 0x7fafec0d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.035+0000 7fafeaffd640 1 --2- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e3d0 0x7fafec0d4030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0097a0 tx=0x7fafdc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e40bc8b3f73da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.035+0000 7fafea7fc640 1 -- 172.21.15.50:0/1051621132 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc035020 con 0x7fafec15e3d0 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.035+0000 7fafea7fc640 1 -- 172.21.15.50:0/1051621132 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc031830 con 0x7fafec15e3d0 2024-04-02T16:22:33.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.035+0000 7fafea7fc640 1 -- 172.21.15.50:0/1051621132 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc031c00 con 0x7fafec15e3d0 2024-04-02T16:22:33.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 -- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e3d0 msgr2=0x7fafec0d4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 --2- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e3d0 0x7fafec0d4030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0097a0 tx=0x7fafdc02b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 -- 172.21.15.50:0/1051621132 shutdown_connections 2024-04-02T16:22:33.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 --2- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e7b0 0x7fafec0d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 --2- 172.21.15.50:0/1051621132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e3d0 0x7fafec0d4030 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 -- 172.21.15.50:0/1051621132 >> 172.21.15.50:0/1051621132 conn(0x7fafec08dc90 msgr2=0x7fafec09d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:33.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 -- 172.21.15.50:0/1051621132 shutdown_connections 2024-04-02T16:22:33.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.036+0000 7faff171c640 1 -- 172.21.15.50:0/1051621132 wait complete. 2024-04-02T16:22:33.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.037+0000 7faff171c640 1 Processor -- start 2024-04-02T16:22:33.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.037+0000 7faff171c640 1 -- start start 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7faff171c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0c7e30 0x7fafec15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7faff171c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e3d0 0x7fafec15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7faff171c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e7b0 0x7fafec15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7faff171c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0d46f0 con 0x7fafec0c7e30 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7faff171c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0d4570 con 0x7fafec15e7b0 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7faff171c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafec0d4870 con 0x7fafec15e3d0 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0c7e30 0x7fafec15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0c7e30 0x7fafec15ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45574/0 (socket says 172.21.15.50:45574) 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 -- 172.21.15.50:0/11058688 learned_addr learned my addr 172.21.15.50:0/11058688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeb7fe640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e7b0 0x7fafec15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafe3fff640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e3d0 0x7fafec15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 -- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e3d0 msgr2=0x7fafec15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e3d0 0x7fafec15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 -- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e7b0 msgr2=0x7fafec15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e7b0 0x7fafec15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 -- 172.21.15.50:0/11058688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec0b8d60 con 0x7fafec0c7e30 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.038+0000 7fafeaffd640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0c7e30 0x7fafec15ca70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fafdc000c00 tx=0x7fafdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.039+0000 7fafe37fe640 1 -- 172.21.15.50:0/11058688 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc035040 con 0x7fafec0c7e30 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.039+0000 7fafe37fe640 1 -- 172.21.15.50:0/11058688 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc004250 con 0x7fafec0c7e30 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.039+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafec0b9040 con 0x7fafec0c7e30 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.039+0000 7fafe37fe640 1 -- 172.21.15.50:0/11058688 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc03a640 con 0x7fafec0c7e30 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.039+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafec0b9600 con 0x7fafec0c7e30 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.040+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafec09f0f0 con 0x7fafec0c7e30 2024-04-02T16:22:33.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.040+0000 7fafe37fe640 1 -- 172.21.15.50:0/11058688 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafdc038020 con 0x7fafec0c7e30 2024-04-02T16:22:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.040+0000 7fafe37fe640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafcc06c8f0 0x7fafcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.040+0000 7fafe37fe640 1 -- 172.21.15.50:0/11058688 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fafdc0adc20 con 0x7fafec0c7e30 2024-04-02T16:22:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.041+0000 7fafe3fff640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafcc06c8f0 0x7fafcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.041+0000 7fafe3fff640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafcc06c8f0 0x7fafcc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ca280 tx=0x7fafd4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.045+0000 7fafe37fe640 1 -- 172.21.15.50:0/11058688 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fafec09f0f0 con 0x7fafec0c7e30 2024-04-02T16:22:33.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.229+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fafec075760 con 0x7fafec0c7e30 2024-04-02T16:22:33.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.229+0000 7fafe37fe640 1 -- 172.21.15.50:0/11058688 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fafdc07bd10 con 0x7fafec0c7e30 2024-04-02T16:22:33.231 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafcc06c8f0 msgr2=0x7fafcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafcc06c8f0 0x7fafcc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ca280 tx=0x7fafd4009040 comp rx=0 tx=0).stop 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0c7e30 msgr2=0x7fafec15ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0c7e30 0x7fafec15ca70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fafdc000c00 tx=0x7fafdc0046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 shutdown_connections 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec15e7b0 0x7fafec15ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec15e3d0 0x7fafec15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafcc06c8f0 0x7fafcc06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 --2- 172.21.15.50:0/11058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0c7e30 0x7fafec15ca70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 >> 172.21.15.50:0/11058688 conn(0x7fafec08dc90 msgr2=0x7fafec162980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 shutdown_connections 2024-04-02T16:22:33.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.231+0000 7faff171c640 1 -- 172.21.15.50:0/11058688 wait complete. 2024-04-02T16:22:33.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:22:33.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.344+0000 7f529dbc8640 1 Processor -- start 2024-04-02T16:22:33.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.344+0000 7f529dbc8640 1 -- start start 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.344+0000 7f529dbc8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f529813dfb0 0x7f52980b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f529dbc8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f529815bf40 0x7f52980ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f529dbc8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529815c350 con 0x7f52980b2100 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f529dbc8640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f529815fb10 con 0x7f529813dfb0 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f529dbc8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f529813e390 con 0x7f529815bf40 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f52977fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f529813dfb0 0x7f52980b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f52977fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f529813dfb0 0x7f52980b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45586/0 (socket says 172.21.15.50:45586) 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f52977fe640 1 -- 172.21.15.50:0/2682001011 learned_addr learned my addr 172.21.15.50:0/2682001011 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:33.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f5297fff640 1 --2- 172.21.15.50:0/2682001011 >> v2:172.21.15.188:3300/0 conn(0x7f529815bf40 0x7f52980ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f52977fe640 1 -- 172.21.15.50:0/2682001011 >> v2:172.21.15.188:3300/0 conn(0x7f529815bf40 msgr2=0x7f52980ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f52977fe640 1 --2- 172.21.15.50:0/2682001011 >> v2:172.21.15.188:3300/0 conn(0x7f529815bf40 0x7f52980ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f52977fe640 1 -- 172.21.15.50:0/2682001011 >> v1:172.21.15.188:6789/0 conn(0x7f52980b2100 legacy=0x7f529815b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f52977fe640 1 -- 172.21.15.50:0/2682001011 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529815ed50 con 0x7f529813dfb0 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.345+0000 7f5297fff640 1 --2- 172.21.15.50:0/2682001011 >> v2:172.21.15.188:3300/0 conn(0x7f529815bf40 0x7f52980ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f52977fe640 1 --2- 172.21.15.50:0/2682001011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f529813dfb0 0x7f52980b1bc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5298159190 tx=0x7f528802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f868420627f03a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f52967fc640 1 -- 172.21.15.50:0/2682001011 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288034070 con 0x7f529813dfb0 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f52967fc640 1 -- 172.21.15.50:0/2682001011 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5288030860 con 0x7f529813dfb0 2024-04-02T16:22:33.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f52967fc640 1 -- 172.21.15.50:0/2682001011 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288030c30 con 0x7f529813dfb0 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 -- 172.21.15.50:0/2682001011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f529813dfb0 msgr2=0x7f52980b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 --2- 172.21.15.50:0/2682001011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f529813dfb0 0x7f52980b1bc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5298159190 tx=0x7f528802b9a0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 -- 172.21.15.50:0/2682001011 shutdown_connections 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 --2- 172.21.15.50:0/2682001011 >> v2:172.21.15.188:3300/0 conn(0x7f529815bf40 0x7f52980ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 --2- 172.21.15.50:0/2682001011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f529813dfb0 0x7f52980b1bc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 -- 172.21.15.50:0/2682001011 >> 172.21.15.50:0/2682001011 conn(0x7f529806ebf0 msgr2=0x7f529806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 -- 172.21.15.50:0/2682001011 shutdown_connections 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.346+0000 7f529dbc8640 1 -- 172.21.15.50:0/2682001011 wait complete. 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 Processor -- start 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 -- start start 2024-04-02T16:22:33.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52980b2100 0x7f52980b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f529813dfb0 0x7f52980b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529815bf40 0x7f5298136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f529815f5f0 con 0x7f52980b2100 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f529815f310 con 0x7f529815bf40 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f529dbc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52980cedd0 con 0x7f529813dfb0 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.348+0000 7f52977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52980b2100 0x7f52980b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52980b2100 0x7f52980b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45592/0 (socket says 172.21.15.50:45592) 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 -- 172.21.15.50:0/1088076841 learned_addr learned my addr 172.21.15.50:0/1088076841 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f5296ffd640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f529813dfb0 0x7f52980b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f5297fff640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529815bf40 0x7f5298136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 -- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f529813dfb0 msgr2=0x7f52980b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f529813dfb0 0x7f52980b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 -- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529815bf40 msgr2=0x7f5298136640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529815bf40 0x7f5298136640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 -- 172.21.15.50:0/1088076841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52981373b0 con 0x7f52980b2100 2024-04-02T16:22:33.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f5297fff640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529815bf40 0x7f5298136640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:33.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f52977fe640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52980b2100 0x7f52980b4110 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f528802b9e0 tx=0x7f5288004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f527ffff640 1 -- 172.21.15.50:0/1088076841 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288048070 con 0x7f52980b2100 2024-04-02T16:22:33.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f527ffff640 1 -- 172.21.15.50:0/1088076841 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5288030660 con 0x7f52980b2100 2024-04-02T16:22:33.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5298137690 con 0x7f52980b2100 2024-04-02T16:22:33.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f527ffff640 1 -- 172.21.15.50:0/1088076841 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288034040 con 0x7f52980b2100 2024-04-02T16:22:33.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.349+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5298137c50 con 0x7f52980b2100 2024-04-02T16:22:33.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.350+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52980756f0 con 0x7f52980b2100 2024-04-02T16:22:33.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.351+0000 7f527ffff640 1 -- 172.21.15.50:0/1088076841 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f528804b070 con 0x7f52980b2100 2024-04-02T16:22:33.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.351+0000 7f527ffff640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f528406c8f0 0x7f528406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.351+0000 7f527ffff640 1 -- 172.21.15.50:0/1088076841 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f52880adf00 con 0x7f52980b2100 2024-04-02T16:22:33.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.351+0000 7f5296ffd640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f528406c8f0 0x7f528406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.352+0000 7f5296ffd640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f528406c8f0 0x7f528406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52981f5690 tx=0x7f528c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.356+0000 7f527ffff640 1 -- 172.21.15.50:0/1088076841 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f528807b9a0 con 0x7f52980b2100 2024-04-02T16:22:33.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.581+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f529815ab80 con 0x7f52980b2100 2024-04-02T16:22:33.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.582+0000 7f527ffff640 1 -- 172.21.15.50:0/1088076841 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f528807fca0 con 0x7f52980b2100 2024-04-02T16:22:33.583 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:33.583 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:22:33.584 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:22:33.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f528406c8f0 msgr2=0x7f528406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f528406c8f0 0x7f528406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52981f5690 tx=0x7f528c009040 comp rx=0 tx=0).stop 2024-04-02T16:22:33.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52980b2100 msgr2=0x7f52980b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52980b2100 0x7f52980b4110 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f528802b9e0 tx=0x7f5288004730 comp rx=0 tx=0).stop 2024-04-02T16:22:33.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 shutdown_connections 2024-04-02T16:22:33.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529815bf40 0x7f5298136640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f529813dfb0 0x7f52980b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f528406c8f0 0x7f528406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 --2- 172.21.15.50:0/1088076841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52980b2100 0x7f52980b4110 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 >> 172.21.15.50:0/1088076841 conn(0x7f529806ebf0 msgr2=0x7f529806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:33.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 shutdown_connections 2024-04-02T16:22:33.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.584+0000 7f529dbc8640 1 -- 172.21.15.50:0/1088076841 wait complete. 2024-04-02T16:22:33.596 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:33.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.697+0000 7fc11b4b7640 1 Processor -- start 2024-04-02T16:22:33.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.697+0000 7fc11b4b7640 1 -- start start 2024-04-02T16:22:33.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc11b4b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc11415e650 0x7fc1140b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc11b4b7640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1140b4c70 con 0x7fc1140b4760 2024-04-02T16:22:33.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc11b4b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1140b4f50 con 0x7fc1140d2e80 2024-04-02T16:22:33.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc11b4b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc1140b5880 con 0x7fc11415e650 2024-04-02T16:22:33.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc118a52640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc1140b4760 0x7fc1140d2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55152/0 (socket says 172.21.15.50:55152) 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc118a52640 1 -- 172.21.15.50:0/2108623196 learned_addr learned my addr 172.21.15.50:0/2108623196 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2018368702 0 0) 0x7fc1140b4c70 con 0x7fc1140b4760 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.698+0000 7fc119253640 1 --2- 172.21.15.50:0/2108623196 >> v2:172.21.15.188:3300/0 conn(0x7fc11415e650 0x7fc1140b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0fc003520 con 0x7fc1140b4760 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 277360299 0 0) 0x7fc0fc003520 con 0x7fc1140b4760 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 >> v2:172.21.15.188:3300/0 conn(0x7fc11415e650 msgr2=0x7fc1140b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 --2- 172.21.15.50:0/2108623196 >> v2:172.21.15.188:3300/0 conn(0x7fc11415e650 0x7fc1140b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 >> v1:172.21.15.188:6789/0 conn(0x7fc1140d2e80 legacy=0x7fc1140cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1140b5230 con 0x7fc1140b4760 2024-04-02T16:22:33.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fc108001670 con 0x7fc1140b4760 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 >> v1:172.21.15.50:6789/0 conn(0x7fc1140b4760 legacy=0x7fc1140d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc0039b0 0x7fc0fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc004800 0x7fc0fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0fc0051c0 0x7fc0fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1140b5880 con 0x7fc0fc0039b0 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00c060 con 0x7fc0fc004800 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc0fc00c260 con 0x7fc0fc0051c0 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119253640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc0039b0 0x7fc0fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc118a52640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc004800 0x7fc0fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119a54640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0fc0051c0 0x7fc0fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119253640 1 -- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0fc0051c0 msgr2=0x7fc0fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:33.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119253640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0fc0051c0 0x7fc0fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119253640 1 -- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc004800 msgr2=0x7fc0fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119253640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc004800 0x7fc0fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119253640 1 -- 172.21.15.50:0/2108623196 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc003520 con 0x7fc0fc0039b0 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.699+0000 7fc119a54640 1 -- 172.21.15.50:0/2108623196 reap_dead start 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.700+0000 7fc119253640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc0039b0 0x7fc0fc003dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009ac0 tx=0x7fc0fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d125fe40e61a28d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.700+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c039070 con 0x7fc0fc0039b0 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.700+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c02c6d0 con 0x7fc0fc0039b0 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.700+0000 7fc113fff640 1 -- 172.21.15.50:0/2108623196 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c009690 con 0x7fc0fc0039b0 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.701+0000 7fc11b4b7640 1 -- 172.21.15.50:0/2108623196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc0039b0 msgr2=0x7fc0fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.701+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc0039b0 0x7fc0fc003dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009ac0 tx=0x7fc0fc005830 comp rx=0 tx=0).stop 2024-04-02T16:22:33.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.701+0000 7fc11b4b7640 1 -- 172.21.15.50:0/2108623196 shutdown_connections 2024-04-02T16:22:33.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.701+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/2108623196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc0039b0 0x7fc0fc003dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.701+0000 7fc11b4b7640 1 -- 172.21.15.50:0/2108623196 >> 172.21.15.50:0/2108623196 conn(0x7fc11409d580 msgr2=0x7fc1140d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:33.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.701+0000 7fc11b4b7640 1 -- 172.21.15.50:0/2108623196 shutdown_connections 2024-04-02T16:22:33.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.701+0000 7fc11b4b7640 1 -- 172.21.15.50:0/2108623196 wait complete. 2024-04-02T16:22:33.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.702+0000 7fc11b4b7640 1 Processor -- start 2024-04-02T16:22:33.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.702+0000 7fc11b4b7640 1 -- start start 2024-04-02T16:22:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc11b4b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc0039b0 0x7fc1140cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc11b4b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1140b9c50 0x7fc114149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc11b4b7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11414c660 0x7fc11414cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc11b4b7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc11415ea30 con 0x7fc11414c660 2024-04-02T16:22:33.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc11b4b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1140d2270 con 0x7fc0fc0039b0 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc11b4b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc11415ebb0 con 0x7fc1140b9c50 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11414c660 0x7fc11414cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11414c660 0x7fc11414cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45608/0 (socket says 172.21.15.50:45608) 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 -- 172.21.15.50:0/3963068290 learned_addr learned my addr 172.21.15.50:0/3963068290 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119253640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc0039b0 0x7fc1140cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc118a52640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1140b9c50 0x7fc114149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 -- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1140b9c50 msgr2=0x7fc114149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1140b9c50 0x7fc114149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 -- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc0039b0 msgr2=0x7fc1140cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc0039b0 0x7fc1140cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 -- 172.21.15.50:0/3963068290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11414a300 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc118a52640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1140b9c50 0x7fc114149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc119a54640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11414c660 0x7fc11414cae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc11408e190 tx=0x7fc10400fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc111ffb640 1 -- 172.21.15.50:0/3963068290 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104003b80 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc111ffb640 1 -- 172.21.15.50:0/3963068290 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc104018070 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.703+0000 7fc111ffb640 1 -- 172.21.15.50:0/3963068290 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1040050d0 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.704+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1140ce5a0 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.704+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1140cdfe0 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.705+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1140756f0 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.705+0000 7fc111ffb640 1 -- 172.21.15.50:0/3963068290 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc104005270 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.705+0000 7fc111ffb640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc10006c920 0x7fc10006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.705+0000 7fc111ffb640 1 -- 172.21.15.50:0/3963068290 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc10408bbc0 con 0x7fc11414c660 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.705+0000 7fc119253640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc10006c920 0x7fc10006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:33.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.706+0000 7fc119253640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc10006c920 0x7fc10006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc10c036000 tx=0x7fc10c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:33.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.710+0000 7fc111ffb640 1 -- 172.21.15.50:0/3963068290 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc104059d40 con 0x7fc11414c660 2024-04-02T16:22:33.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.894+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1140d1aa0 con 0x7fc11414c660 2024-04-02T16:22:33.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.895+0000 7fc111ffb640 1 -- 172.21.15.50:0/3963068290 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fc104059640 con 0x7fc11414c660 2024-04-02T16:22:33.896 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:33.896 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":25,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:07.424718+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:33.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc10006c920 msgr2=0x7fc10006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc10006c920 0x7fc10006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc10c036000 tx=0x7fc10c030000 comp rx=0 tx=0).stop 2024-04-02T16:22:33.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11414c660 msgr2=0x7fc11414cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:33.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11414c660 0x7fc11414cae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc11408e190 tx=0x7fc10400fc70 comp rx=0 tx=0).stop 2024-04-02T16:22:33.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 shutdown_connections 2024-04-02T16:22:33.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc10006c920 0x7fc10006ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11414c660 0x7fc11414cae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1140b9c50 0x7fc114149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 --2- 172.21.15.50:0/3963068290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0fc0039b0 0x7fc1140cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 >> 172.21.15.50:0/3963068290 conn(0x7fc11409d580 msgr2=0x7fc1140d3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 shutdown_connections 2024-04-02T16:22:33.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:33.897+0000 7fc11b4b7640 1 -- 172.21.15.50:0/3963068290 wait complete. 2024-04-02T16:22:33.909 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:22:34.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.010+0000 7f633f0f4640 1 Processor -- start 2024-04-02T16:22:34.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.010+0000 7f633f0f4640 1 -- start start 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f633f0f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f63380d2f10 0x7f63380cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f633f0f4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63380ceda0 con 0x7f63380d0e90 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f633f0f4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63380cf040 con 0x7f633815e510 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f633f0f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f63380d19c0 con 0x7f63380d2f10 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f633d691640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f63380d0e90 0x7f63380d12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55154/0 (socket says 172.21.15.50:55154) 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f633d691640 1 -- 172.21.15.50:0/1660651453 learned_addr learned my addr 172.21.15.50:0/1660651453 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f6337fff640 1 --2- 172.21.15.50:0/1660651453 >> v2:172.21.15.188:3300/0 conn(0x7f63380d2f10 0x7f63380cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638856137 0 0) 0x7f63380ceda0 con 0x7f63380d0e90 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6328003520 con 0x7f63380d0e90 2024-04-02T16:22:34.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3753800897 0 0) 0x7f6328003520 con 0x7f63380d0e90 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 >> v2:172.21.15.188:3300/0 conn(0x7f63380d2f10 msgr2=0x7f63380cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 --2- 172.21.15.50:0/1660651453 >> v2:172.21.15.188:3300/0 conn(0x7f63380d2f10 0x7f63380cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 >> v1:172.21.15.188:6789/0 conn(0x7f633815e510 legacy=0x7f63380d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63380cf320 con 0x7f63380d0e90 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f6330001a10 con 0x7f63380d0e90 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 >> v1:172.21.15.50:6789/0 conn(0x7f63380d0e90 legacy=0x7f63380d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280039b0 0x7f6328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f6328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63280051c0 0x7f632800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63380d19c0 con 0x7f6328004800 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f632800c160 con 0x7f63280051c0 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f632800c360 con 0x7f63280039b0 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.011+0000 7f6337fff640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f6328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f633d691640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63280051c0 0x7f632800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f633ce90640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280039b0 0x7f6328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f6337fff640 1 -- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280039b0 msgr2=0x7f6328003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f6337fff640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280039b0 0x7f6328003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f6337fff640 1 -- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63280051c0 msgr2=0x7f632800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f6337fff640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63280051c0 0x7f632800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f633d691640 1 -- 172.21.15.50:0/1660651453 reap_dead start 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f6337fff640 1 -- 172.21.15.50:0/1660651453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6328003520 con 0x7f6328004800 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f633ce90640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280039b0 0x7f6328003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.012+0000 7f6337fff640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f6328004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6324009a90 tx=0x7f6328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f8c3f73431640d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324039070 con 0x7f6328004800 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632402c680 con 0x7f6328004800 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f63377fe640 1 -- 172.21.15.50:0/1660651453 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324009690 con 0x7f6328004800 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f633f0f4640 1 -- 172.21.15.50:0/1660651453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 msgr2=0x7f6328004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f633f0f4640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f6328004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6324009a90 tx=0x7f6328005830 comp rx=0 tx=0).stop 2024-04-02T16:22:34.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f633f0f4640 1 -- 172.21.15.50:0/1660651453 shutdown_connections 2024-04-02T16:22:34.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f633f0f4640 1 --2- 172.21.15.50:0/1660651453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f6328004c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f633f0f4640 1 -- 172.21.15.50:0/1660651453 >> 172.21.15.50:0/1660651453 conn(0x7f633809d570 msgr2=0x7f63380d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f633f0f4640 1 -- 172.21.15.50:0/1660651453 shutdown_connections 2024-04-02T16:22:34.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.013+0000 7f633f0f4640 1 -- 172.21.15.50:0/1660651453 wait complete. 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.014+0000 7f633f0f4640 1 Processor -- start 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.014+0000 7f633f0f4640 1 -- start start 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633f0f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f63380b9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633f0f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63380ce0c0 0x7f63381f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633f0f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63381f47f0 0x7f63381f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633f0f4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63380cfe00 con 0x7f6328004800 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633f0f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f633815e8f0 con 0x7f63381f47f0 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633f0f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f63380cff80 con 0x7f63380ce0c0 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633ce90640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f63380b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633ce90640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f63380b9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45620/0 (socket says 172.21.15.50:45620) 2024-04-02T16:22:34.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.015+0000 7f633ce90640 1 -- 172.21.15.50:0/3573591357 learned_addr learned my addr 172.21.15.50:0/3573591357 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633d691640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63381f47f0 0x7f63381f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f6337fff640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63380ce0c0 0x7f63381f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633ce90640 1 -- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63380ce0c0 msgr2=0x7f63381f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633ce90640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63380ce0c0 0x7f63381f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633ce90640 1 -- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63381f47f0 msgr2=0x7f63381f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633ce90640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63381f47f0 0x7f63381f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633ce90640 1 -- 172.21.15.50:0/3573591357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633815cde0 con 0x7f6328004800 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633ce90640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f63380b9420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e480 tx=0x7f632c00e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f63357fa640 1 -- 172.21.15.50:0/3573591357 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c019070 con 0x7f6328004800 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f63357fa640 1 -- 172.21.15.50:0/3573591357 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632c0052e0 con 0x7f6328004800 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f63357fa640 1 -- 172.21.15.50:0/3573591357 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c003830 con 0x7f6328004800 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63381e6f30 con 0x7f6328004800 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.016+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63380b8940 con 0x7f6328004800 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.017+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63380756f0 con 0x7f6328004800 2024-04-02T16:22:34.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.017+0000 7f63357fa640 1 -- 172.21.15.50:0/3573591357 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f632c01c070 con 0x7f6328004800 2024-04-02T16:22:34.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.018+0000 7f63357fa640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f631c06c920 0x7f631c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.018+0000 7f63357fa640 1 -- 172.21.15.50:0/3573591357 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f632c08b440 con 0x7f6328004800 2024-04-02T16:22:34.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.018+0000 7f6337fff640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f631c06c920 0x7f631c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.019+0000 7f6337fff640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f631c06c920 0x7f631c06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6324036000 tx=0x7f6324030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.022+0000 7f63357fa640 1 -- 172.21.15.50:0/3573591357 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f632c058ee0 con 0x7f6328004800 2024-04-02T16:22:34.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.245+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63381612c0 con 0x7f6328004800 2024-04-02T16:22:34.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.246+0000 7f63357fa640 1 -- 172.21.15.50:0/3573591357 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f632c05d1e0 con 0x7f6328004800 2024-04-02T16:22:34.246 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:34.246 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:22:34.247 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:22:34.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f631c06c920 msgr2=0x7f631c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f631c06c920 0x7f631c06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6324036000 tx=0x7f6324030000 comp rx=0 tx=0).stop 2024-04-02T16:22:34.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 msgr2=0x7f63380b9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f63380b9420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e480 tx=0x7f632c00e880 comp rx=0 tx=0).stop 2024-04-02T16:22:34.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 shutdown_connections 2024-04-02T16:22:34.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63381f47f0 0x7f63381f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63380ce0c0 0x7f63381f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f631c06c920 0x7f631c06ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 --2- 172.21.15.50:0/3573591357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6328004800 0x7f63380b9420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 >> 172.21.15.50:0/3573591357 conn(0x7f633809d570 msgr2=0x7f63380ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 shutdown_connections 2024-04-02T16:22:34.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.248+0000 7f633f0f4640 1 -- 172.21.15.50:0/3573591357 wait complete. 2024-04-02T16:22:34.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T16:22:00.747313+0000', 'modified': '2024-04-02T16:22:22.833211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6837/2647829095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 2647829095}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 2647829095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-04-02T16:22:34.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T16:22:34.662 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T16:22:34.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.771+0000 7f4d4f247640 1 Processor -- start 2024-04-02T16:22:34.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.771+0000 7f4d4f247640 1 -- start start 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d4f247640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d480d1390 0x7f4d48162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d4f247640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d480d5000 con 0x7f4d480d4980 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d4f247640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d480cf0d0 con 0x7f4d48161ca0 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d4f247640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4d481630f0 con 0x7f4d480d1390 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d47fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f4d480d4980 0x7f4d480d0c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55168/0 (socket says 172.21.15.50:55168) 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d47fff640 1 -- 172.21.15.50:0/374225855 learned_addr learned my addr 172.21.15.50:0/374225855 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905789991 0 0) 0x7f4d480d5000 con 0x7f4d480d4980 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d38003520 con 0x7f4d480d4980 2024-04-02T16:22:34.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/374225855 >> v2:172.21.15.188:3300/0 conn(0x7f4d480d1390 0x7f4d48162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1326185949 0 0) 0x7f4d38003520 con 0x7f4d480d4980 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 >> v2:172.21.15.188:3300/0 conn(0x7f4d480d1390 msgr2=0x7f4d48162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 --2- 172.21.15.50:0/374225855 >> v2:172.21.15.188:3300/0 conn(0x7f4d480d1390 0x7f4d48162bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 >> v1:172.21.15.188:6789/0 conn(0x7f4d48161ca0 legacy=0x7f4d480cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d480cf3b0 con 0x7f4d480d4980 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f4d34001690 con 0x7f4d480d4980 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 >> v1:172.21.15.50:6789/0 conn(0x7f4d480d4980 legacy=0x7f4d480d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380039b0 0x7f4d38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d38004800 0x7f4d38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.772+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481630f0 con 0x7f4d380052e0 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3800c1b0 con 0x7f4d380039b0 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3800c3b0 con 0x7f4d38004800 2024-04-02T16:22:34.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380039b0 0x7f4d38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d47fff640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d38004800 0x7f4d38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d4d7e4640 1 -- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d38004800 msgr2=0x7f4d38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d38004800 0x7f4d38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d4d7e4640 1 -- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380039b0 msgr2=0x7f4d38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380039b0 0x7f4d38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d4d7e4640 1 -- 172.21.15.50:0/374225855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d38003520 con 0x7f4d380052e0 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.773+0000 7f4d47fff640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d38004800 0x7f4d38004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4d7e4640 1 -- 172.21.15.50:0/374225855 reap_dead start 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d3800b9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009f90 tx=0x7f4d38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23331a5b536ce51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40009c10 con 0x7f4d380052e0 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4002cc20 con 0x7f4d380052e0 2024-04-02T16:22:34.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d477fe640 1 -- 172.21.15.50:0/374225855 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40003990 con 0x7f4d380052e0 2024-04-02T16:22:34.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4f247640 1 -- 172.21.15.50:0/374225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 msgr2=0x7f4d3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4f247640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d3800b9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009f90 tx=0x7f4d38005830 comp rx=0 tx=0).stop 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4f247640 1 -- 172.21.15.50:0/374225855 shutdown_connections 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4f247640 1 --2- 172.21.15.50:0/374225855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d3800b9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4f247640 1 -- 172.21.15.50:0/374225855 >> 172.21.15.50:0/374225855 conn(0x7f4d4809d5c0 msgr2=0x7f4d480d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4f247640 1 -- 172.21.15.50:0/374225855 shutdown_connections 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.774+0000 7f4d4f247640 1 -- 172.21.15.50:0/374225855 wait complete. 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 Processor -- start 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 -- start start 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d480b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d480cc390 0x7f4d480b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d481f7070 0x7f4d481f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48162bb0 con 0x7f4d380052e0 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d480ced80 con 0x7f4d480cc390 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.776+0000 7f4d4f247640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d48162d30 con 0x7f4d481f7070 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d480b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d480b9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45636/0 (socket says 172.21.15.50:45636) 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4215424883 learned_addr learned my addr 172.21.15.50:0/4215424883 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d481f7070 0x7f4d481f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d47fff640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d480cc390 0x7f4d480b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d481f7070 msgr2=0x7f4d481f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d481f7070 0x7f4d481f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d480cc390 msgr2=0x7f4d480b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d480cc390 0x7f4d480b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4215424883 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d480cdc80 con 0x7f4d380052e0 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d481f7070 0x7f4d481f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d480b9940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00f620 tx=0x7f4d3c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d457fa640 1 -- 172.21.15.50:0/4215424883 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0052e0 con 0x7f4d380052e0 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d457fa640 1 -- 172.21.15.50:0/4215424883 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c018070 con 0x7f4d380052e0 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d457fa640 1 -- 172.21.15.50:0/4215424883 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c003810 con 0x7f4d380052e0 2024-04-02T16:22:34.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d480ccbe0 con 0x7f4d380052e0 2024-04-02T16:22:34.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.777+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d480cd1a0 con 0x7f4d380052e0 2024-04-02T16:22:34.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.779+0000 7f4d457fa640 1 -- 172.21.15.50:0/4215424883 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d3c0039b0 con 0x7f4d380052e0 2024-04-02T16:22:34.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.779+0000 7f4d457fa640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.779+0000 7f4d457fa640 1 -- 172.21.15.50:0/4215424883 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4d3c08bb10 con 0x7f4d380052e0 2024-04-02T16:22:34.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.779+0000 7f4d47fff640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.780+0000 7f4d47fff640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d480d2ae0 tx=0x7f4d34007040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.780+0000 7f4d51e2b740 1 Processor -- start 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.780+0000 7f4d51e2b740 1 -- start start 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9eb90 0x560d56b71b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b72070 0x560d56ba1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560d56adb060 con 0x560d56b9ef70 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560d56a83650 con 0x560d56b72070 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560d56ad7010 con 0x560d56b9eb90 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45650/0 (socket says 172.21.15.50:45650) 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 -- 172.21.15.50:0/346046019 learned_addr learned my addr 172.21.15.50:0/346046019 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9eb90 0x560d56b71b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b72070 0x560d56ba1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 -- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9eb90 msgr2=0x560d56b71b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9eb90 0x560d56b71b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 -- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b72070 msgr2=0x560d56ba1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b72070 0x560d56ba1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 -- 172.21.15.50:0/346046019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d567ef2d0 con 0x560d56b9ef70 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d47fff640 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba1920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d34011c60 tx=0x7f4d34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728eb194221fce88 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d327fc640 1 -- 172.21.15.50:0/346046019 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34005260 con 0x560d56b9ef70 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.781+0000 7f4d327fc640 1 -- 172.21.15.50:0/346046019 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34005400 con 0x560d56b9ef70 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d327fc640 1 -- 172.21.15.50:0/346046019 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34015470 con 0x560d56b9ef70 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 -- 172.21.15.50:0/346046019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 msgr2=0x560d56ba1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba1920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d34011c60 tx=0x7f4d34014000 comp rx=0 tx=0).stop 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 -- 172.21.15.50:0/346046019 shutdown_connections 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b72070 0x560d56ba1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba1920 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/346046019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9eb90 0x560d56b71b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 -- 172.21.15.50:0/346046019 >> 172.21.15.50:0/346046019 conn(0x560d56b73da0 msgr2=0x560d56bbc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 -- 172.21.15.50:0/346046019 shutdown_connections 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.782+0000 7f4d51e2b740 1 -- 172.21.15.50:0/346046019 wait complete. 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.783+0000 7f4d51e2b740 1 -- start start 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.783+0000 7f4d51e2b740 1 Processor -- start 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b9eb90 0x560d56bcc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56ba33c0 0x560d56ba8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560d56b76440 con 0x560d56b9ef70 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560d56a96a10 con 0x560d56b9eb90 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560d56b78c80 con 0x560d56ba33c0 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 10 client.0 _ll_register_callbacks cb 0x560d56ad30b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45662/0 (socket says 172.21.15.50:45662) 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 -- 172.21.15.50:0/1842400112 learned_addr learned my addr 172.21.15.50:0/1842400112 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b9eb90 0x560d56bcc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56ba33c0 0x560d56ba8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 -- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56ba33c0 msgr2=0x560d56ba8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56ba33c0 0x560d56ba8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 -- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b9eb90 msgr2=0x560d56bcc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b9eb90 0x560d56bcc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 -- 172.21.15.50:0/1842400112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d568b96b0 con 0x560d56b9ef70 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d47fff640 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba2e80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4d34025040 tx=0x7f4d34001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb513fe5b931f7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d277fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d277fe640 1 -- 172.21.15.50:0/1842400112 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34030020 con 0x560d56b9ef70 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d277fe640 1 -- 172.21.15.50:0/1842400112 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4d34015c90 con 0x560d56b9ef70 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.784+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d34005400 con 0x560d56b9ef70 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d277fe640 1 -- 172.21.15.50:0/1842400112 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f4d3403a050 con 0x560d56b9ef70 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 10 client.4866 fetch_fsmap learned FSMap version 9 2024-04-02T16:22:34.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x560d56ba4170 con 0x560d56b9ef70 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d277fe640 1 -- 172.21.15.50:0/1842400112 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34005400 con 0x560d56b9ef70 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d277fe640 1 -- 172.21.15.50:0/1842400112 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f4d3403d020 con 0x560d56b9ef70 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 10 client.4866 fetch_fsmap finished waiting for FSMap version 9 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 10 client.4866 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 20 client.4866 populate_metadata read hostname 'smithi050' 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56bd0a70 0x560d56ba6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 4 client.4866 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 --> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d56a82cf0 con 0x560d56bd0a70 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.785+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56bd0a70 0x560d56ba6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.786+0000 7f4d277fe640 10 client.4866 ms_handle_connect on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.786+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56bd0a70 0x560d56ba6ce0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=516eca45bb89a45d server_cookie=345f075fc43efac5 in_seq=0 out_seq=0 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.786+0000 7f4d277fe640 1 -- 172.21.15.50:0/1842400112 <== mds.0 v2:172.21.15.188:6834/2647829095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d56a82cf0 con 0x560d56bd0a70 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.786+0000 7f4d277fe640 10 client.4866 handle_command_reply: tid=0 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 client.4866 shutdown 2024-04-02T16:22:34.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 msgr2=0x560d56ba2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba2e80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4d34025040 tx=0x7f4d34001150 comp rx=0 tx=0).stop 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 shutdown_connections 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56bd0a70 0x560d56ba6ce0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56ba33c0 0x560d56ba8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9ef70 0x560d56ba2e80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/1842400112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56b9eb90 0x560d56bcc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d277fe640 0 client.4866 ms_handle_reset on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 >> 172.21.15.50:0/1842400112 conn(0x560d56b72070 msgr2=0x560d56b75a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 shutdown_connections 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 1 -- 172.21.15.50:0/1842400112 wait complete. 2024-04-02T16:22:34.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.787+0000 7f4d51e2b740 20 client.4866 trim_cache size 0 max 16384 2024-04-02T16:22:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.805+0000 7f4d51e2b740 1 Processor -- start 2024-04-02T16:22:34.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.805+0000 7f4d51e2b740 1 -- start start 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56b77d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56c2dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48201480 con 0x560d56b9eb90 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d482007e0 con 0x560d56ba33c0 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d48200e30 con 0x560d56b9ef70 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56b77d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56b77d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45670/0 (socket says 172.21.15.50:45670) 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4092771989 learned_addr learned my addr 172.21.15.50:0/4092771989 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d47fff640 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56c2dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 msgr2=0x560d56c2dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56c2dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 msgr2=0x560d56bbca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/4092771989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d56bccad0 con 0x560d56b9eb90 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56b77d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c059290 tx=0x7f4d3c06f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6653f0de0886eb61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d31ffb640 1 -- 172.21.15.50:0/4092771989 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c05d940 con 0x560d56b9eb90 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d31ffb640 1 -- 172.21.15.50:0/4092771989 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c088070 con 0x560d56b9eb90 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.806+0000 7f4d31ffb640 1 -- 172.21.15.50:0/4092771989 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c062170 con 0x560d56b9eb90 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 -- 172.21.15.50:0/4092771989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 msgr2=0x560d56b77d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56b77d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c059290 tx=0x7f4d3c06f040 comp rx=0 tx=0).stop 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 -- 172.21.15.50:0/4092771989 shutdown_connections 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56c2dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/4092771989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56b77d20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 -- 172.21.15.50:0/4092771989 >> 172.21.15.50:0/4092771989 conn(0x560d56b72070 msgr2=0x560d56ba6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 -- 172.21.15.50:0/4092771989 shutdown_connections 2024-04-02T16:22:34.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 -- 172.21.15.50:0/4092771989 wait complete. 2024-04-02T16:22:34.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.807+0000 7f4d51e2b740 1 -- start start 2024-04-02T16:22:34.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 1 Processor -- start 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56c19a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56bbca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560d56bd08f0 con 0x560d56b9eb90 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560d56ba4450 con 0x560d56ba33c0 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560d56bd0f40 con 0x560d56b9ef70 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d51e2b740 10 client.0 _ll_register_callbacks cb 0x7f4d380036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.808+0000 7f4d4cfe3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56c19a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56c19a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45682/0 (socket says 172.21.15.50:45682) 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/278989768 learned_addr learned my addr 172.21.15.50:0/278989768 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:34.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d47fff640 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56bbca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4d7e4640 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 msgr2=0x560d56bbca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56bbca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 msgr2=0x560d56bbcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 -- 172.21.15.50:0/278989768 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d56be02c0 con 0x560d56b9eb90 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d4cfe3640 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56c19a60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000fb0 tx=0x7f4d3c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65e33e3ebcc7ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d277fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d277fe640 1 -- 172.21.15.50:0/278989768 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c07c070 con 0x560d56b9eb90 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d277fe640 1 -- 172.21.15.50:0/278989768 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4d3c055c00 con 0x560d56b9eb90 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c088070 con 0x560d56b9eb90 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d277fe640 1 -- 172.21.15.50:0/278989768 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c05d940 con 0x560d56b9eb90 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d51e2b740 10 client.4872 fetch_fsmap learned FSMap version 9 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x560d56c2b860 con 0x560d56b9eb90 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d277fe640 1 -- 172.21.15.50:0/278989768 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c088070 con 0x560d56b9eb90 2024-04-02T16:22:34.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d277fe640 1 -- 172.21.15.50:0/278989768 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f4d3c003ca0 con 0x560d56b9eb90 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d51e2b740 10 client.4872 fetch_fsmap finished waiting for FSMap version 9 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.809+0000 7f4d51e2b740 10 client.4872 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d51e2b740 20 client.4872 populate_metadata read hostname 'smithi050' 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56c3ab40 0x560d56c3af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d51e2b740 4 client.4872 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 --> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] -- command(tid 0: {"prefix": "status"}) -- 0x560d56baa5d0 con 0x560d56c3ab40 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d47fff640 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56c3ab40 0x560d56c3af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d47fff640 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56c3ab40 0x560d56c3af20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5869147956acd6 server_cookie=4b6bd9547d6463e3 in_seq=0 out_seq=0 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d277fe640 10 client.4872 ms_handle_connect on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d277fe640 1 -- 172.21.15.50:0/278989768 <== mds.0 v2:172.21.15.188:6834/2647829095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x560d56baa5d0 con 0x560d56c3ab40 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d277fe640 10 client.4872 handle_command_reply: tid=0 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.810+0000 7f4d51e2b740 1 client.4872 shutdown 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 msgr2=0x560d56c19a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56c19a60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000fb0 tx=0x7f4d3c05c040 comp rx=0 tx=0).stop 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 shutdown_connections 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:6834/2647829095,v1:172.21.15.188:6837/2647829095] conn(0x560d56c3ab40 0x560d56c3af20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d56ba33c0 0x560d56bbcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560d56b9ef70 0x560d56bbca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d277fe640 0 client.4872 ms_handle_reset on v2:172.21.15.188:6834/2647829095 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 --2- 172.21.15.50:0/278989768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d56b9eb90 0x560d56c19a60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 >> 172.21.15.50:0/278989768 conn(0x560d56b72070 msgr2=0x560d56b9df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 shutdown_connections 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 1 -- 172.21.15.50:0/278989768 wait complete. 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d51e2b740 20 client.4872 trim_cache size 0 max 16384 2024-04-02T16:22:34.820 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "057e6c06-7c98-48a0-9312-9e11e9abe7fe", 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 4294, 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 33.407678017999999, 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 9, 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 25, 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 25, 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 40.569757097 2024-04-02T16:22:34.821 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4d2c06c8f0 msgr2=0x7f4d2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d480d2ae0 tx=0x7f4d34007040 comp rx=0 tx=0).stop 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 msgr2=0x7f4d480b9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d480b9940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00f620 tx=0x7f4d3c00fae0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 shutdown_connections 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d481f7070 0x7f4d481f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d480cc390 0x7f4d480b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 --2- 172.21.15.50:0/4215424883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d380052e0 0x7f4d480b9940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 >> 172.21.15.50:0/4215424883 conn(0x7f4d4809d5c0 msgr2=0x7f4d480d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:34.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 shutdown_connections 2024-04-02T16:22:34.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:34.811+0000 7f4d4f247640 1 -- 172.21.15.50:0/4215424883 wait complete. 2024-04-02T16:22:34.824 INFO:teuthology.run_tasks:Running task sequential... 2024-04-02T16:22:34.833 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-02T16:22:34.834 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-02T16:22:34.834 INFO:teuthology.task.print:Enabling mgr modules 2024-04-02T16:22:34.834 INFO:teuthology.run_tasks:Running task kclient... 2024-04-02T16:22:34.845 INFO:tasks.kclient:Mounting kernel clients... 2024-04-02T16:22:34.846 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-02T16:22:34.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:34.846 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-04-02T16:22:34.860 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-02T16:22:34.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:34.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T16:22:34.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:34.928 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-04-02T16:22:35.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.040 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-04-02T16:22:35.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-04-02T16:22:35.126 INFO:teuthology.orchestra.run.smithi050.stderr:Cannot find device "ceph-brx" 2024-04-02T16:22:35.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:22:35.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.128 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-02T16:22:35.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-04-02T16:22:35.222 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-04-02T16:22:35.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:22:35.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.224 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-02T16:22:35.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.281 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-04-02T16:22:35.352 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-04-02T16:22:35.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:22:35.354 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-02T16:22:35.354 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T16:22:35.354 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T16:22:35.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:22:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.457+0000 7f6222aeb640 1 Processor -- start 2024-04-02T16:22:35.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.457+0000 7f6222aeb640 1 -- start start 2024-04-02T16:22:35.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.458+0000 7f6222aeb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:35.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.458+0000 7f6222aeb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f621c0b2100 0x7f621c15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:35.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.458+0000 7f6222aeb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621c15faf0 con 0x7f621c15be10 2024-04-02T16:22:35.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.458+0000 7f6222aeb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f621c15ed30 con 0x7f621c0b2100 2024-04-02T16:22:35.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.458+0000 7f6222aeb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f621c15f010 con 0x7f621c13deb0 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.458+0000 7f6221088640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f621c15be10 0x7f621c0b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:55184/0 (socket says 172.21.15.50:55184) 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.458+0000 7f6221088640 1 -- 172.21.15.50:0/3702736588 learned_addr learned my addr 172.21.15.50:0/3702736588 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621bfff640 1 --2- 172.21.15.50:0/3702736588 >> v2:172.21.15.188:3300/0 conn(0x7f621c0b2100 0x7f621c15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f6220887640 1 --2- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621b7fe640 1 -- 172.21.15.50:0/3702736588 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537927547 0 0) 0x7f621c15faf0 con 0x7f621c15be10 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621b7fe640 1 -- 172.21.15.50:0/3702736588 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f620c003520 con 0x7f621c15be10 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621bfff640 1 -- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 msgr2=0x7f621c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621bfff640 1 --2- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621b7fe640 1 -- 172.21.15.50:0/3702736588 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1013721044 0 0) 0x7f620c003520 con 0x7f621c15be10 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621bfff640 1 -- 172.21.15.50:0/3702736588 >> v1:172.21.15.50:6789/0 conn(0x7f621c15be10 legacy=0x7f621c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f621bfff640 1 -- 172.21.15.50:0/3702736588 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c15f2f0 con 0x7f621c0b2100 2024-04-02T16:22:35.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.459+0000 7f6220887640 1 --2- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f621bfff640 1 --2- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c0b2100 0x7f621c15b8d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f62080097a0 tx=0x7f620802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e770745b7235a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f621b7fe640 1 -- 172.21.15.50:0/3702736588 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208037070 con 0x7f621c0b2100 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f621b7fe640 1 -- 172.21.15.50:0/3702736588 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6208030510 con 0x7f621c0b2100 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f621b7fe640 1 -- 172.21.15.50:0/3702736588 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140019f0 con 0x7f621c0b2100 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f6222aeb640 1 -- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c0b2100 msgr2=0x7f621c15b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c0b2100 0x7f621c15b8d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f62080097a0 tx=0x7f620802b790 comp rx=0 tx=0).stop 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f6222aeb640 1 -- 172.21.15.50:0/3702736588 shutdown_connections 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c0b2100 0x7f621c15b8d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3702736588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f6222aeb640 1 -- 172.21.15.50:0/3702736588 >> 172.21.15.50:0/3702736588 conn(0x7f621c06ebf0 msgr2=0x7f621c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:35.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.460+0000 7f6222aeb640 1 -- 172.21.15.50:0/3702736588 shutdown_connections 2024-04-02T16:22:35.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.461+0000 7f6222aeb640 1 -- 172.21.15.50:0/3702736588 wait complete. 2024-04-02T16:22:35.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.462+0000 7f6222aeb640 1 Processor -- start 2024-04-02T16:22:35.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.462+0000 7f6222aeb640 1 -- start start 2024-04-02T16:22:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.462+0000 7f6222aeb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f621c0b2100 0x7f621c158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.462+0000 7f6222aeb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c137750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6222aeb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c15be10 0x7f621c138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:35.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6222aeb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f621c15c420 con 0x7f621c0b2100 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6222aeb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f621c15c2a0 con 0x7f621c15be10 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6222aeb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f621c13e290 con 0x7f621c13deb0 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f621c0b2100 0x7f621c158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f621c0b2100 0x7f621c158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45696/0 (socket says 172.21.15.50:45696) 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 -- 172.21.15.50:0/3342894844 learned_addr learned my addr 172.21.15.50:0/3342894844 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6221088640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c15be10 0x7f621c138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 -- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 msgr2=0x7f621c137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f621bfff640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c137750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c137750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 -- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c15be10 msgr2=0x7f621c138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c15be10 0x7f621c138df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 -- 172.21.15.50:0/3342894844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c0b0390 con 0x7f621c0b2100 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6221088640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c15be10 0x7f621c138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:35.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f621bfff640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6220887640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f621c0b2100 0x7f621c158710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f621000ebe0 tx=0x7f621000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f62197fa640 1 -- 172.21.15.50:0/3342894844 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210017070 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f62197fa640 1 -- 172.21.15.50:0/3342894844 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621000cc30 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c0b0630 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f62197fa640 1 -- 172.21.15.50:0/3342894844 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210013470 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.463+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c0b0bf0 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.464+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c0756f0 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.465+0000 7f62197fa640 1 -- 172.21.15.50:0/3342894844 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f621001a070 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.465+0000 7f62197fa640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f620006c8f0 0x7f620006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.465+0000 7f62197fa640 1 -- 172.21.15.50:0/3342894844 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f621008be10 con 0x7f621c0b2100 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.465+0000 7f621bfff640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f620006c8f0 0x7f620006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:35.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.466+0000 7f621bfff640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f620006c8f0 0x7f620006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6208034000 tx=0x7f620802f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:35.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.469+0000 7f62197fa640 1 -- 172.21.15.50:0/3342894844 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6210059f00 con 0x7f621c0b2100 2024-04-02T16:22:35.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.694+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f621c0b3630 con 0x7f621c0b2100 2024-04-02T16:22:35.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.694+0000 7f62197fa640 1 -- 172.21.15.50:0/3342894844 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f62100600d0 con 0x7f621c0b2100 2024-04-02T16:22:35.698 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f620006c8f0 msgr2=0x7f620006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f620006c8f0 0x7f620006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6208034000 tx=0x7f620802f000 comp rx=0 tx=0).stop 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f621c0b2100 msgr2=0x7f621c158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f621c0b2100 0x7f621c158710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f621000ebe0 tx=0x7f621000c520 comp rx=0 tx=0).stop 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 shutdown_connections 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f621c15be10 0x7f621c138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f621c13deb0 0x7f621c137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f620006c8f0 0x7f620006edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 --2- 172.21.15.50:0/3342894844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f621c0b2100 0x7f621c158710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 >> 172.21.15.50:0/3342894844 conn(0x7f621c06ebf0 msgr2=0x7f621c15c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:35.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 shutdown_connections 2024-04-02T16:22:35.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35.696+0000 7f6222aeb640 1 -- 172.21.15.50:0/3342894844 wait complete. 2024-04-02T16:22:35.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:22:35.708 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:22:35.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.709 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-04-02T16:22:35.723 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-04-02T16:22:35.724 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42694sec preferred_lft 42694sec 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:35.725 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-04-02T16:22:35.725 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T16:22:35.726 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:35.726 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:35.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-04-02T16:22:35.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-04-02T16:22:35.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-04-02T16:22:35.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T16:22:35.726 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:35.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:35.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:35.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.871 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:22:35.897 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-04-02T16:22:35.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:35.898 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-04-02T16:22:35.953 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-04-02T16:22:35.953 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:22:35.953 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T16:22:35.953 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T16:22:35.954 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:22:35.954 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:35.954 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:35.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T16:22:35.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T16:22:35.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T16:22:35.954 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:36.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:36.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:36.089 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-04-02T16:22:36.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:36.107 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-04-02T16:22:36.162 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:36.163 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:36.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:22:36.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T16:22:36.163 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:36.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:36.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:36.284 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T16:22:36.284 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:36.284 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:36.284 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T16:22:36.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T16:22:36.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T16:22:36.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T16:22:36.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T16:22:36.285 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:36.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:36.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:36.408 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:36.408 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:36.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-04-02T16:22:36.408 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T16:22:36.408 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:36.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:36.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:36.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:36.490 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T16:22:36.504 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:22:36.504 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:36.504 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:22:36.559 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-02T16:22:36.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:36.559 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:22:36.644 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:22:36.644 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-02T16:22:36.644 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:22:36.766 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:22:36.766 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-02T16:22:36.767 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:22:36.767 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:36.767 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-02T16:22:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=057e6c06-7c98-48a0-9312-9e11e9abe7fe" will pass to kernel 2024-04-02T16:22:36.773 INFO:teuthology.orchestra.run.smithi050.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-02T16:22:36.773 INFO:teuthology.orchestra.run.smithi050.stdout: You just mounted a file system that supports labels which does not 2024-04-02T16:22:36.773 INFO:teuthology.orchestra.run.smithi050.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T16:22:36.773 INFO:teuthology.orchestra.run.smithi050.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T16:22:36.773 INFO:teuthology.orchestra.run.smithi050.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T16:22:36.775 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T16:22:36.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:36.775 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:22:36.802 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:36.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:36.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:36.897 INFO:teuthology.orchestra.run.smithi050.stdout:client4756 2024-04-02T16:22:36.898 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:36.898 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:36.913 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:37.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:37.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:37.007 INFO:teuthology.orchestra.run.smithi050.stdout:client4756 2024-04-02T16:22:37.008 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:37.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:37.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:37.064 INFO:teuthology.orchestra.run.smithi050.stdout:client4756 2024-04-02T16:22:37.064 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/057e6c06-7c98-48a0-9312-9e11e9abe7fe.client4756/status') 2024-04-02T16:22:37.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:37.126 INFO:teuthology.orchestra.run.smithi050.stdout:instance: client.4756 (3)172.21.15.50:0/3539324361 2024-04-02T16:22:37.126 INFO:teuthology.orchestra.run.smithi050.stdout:blocklisted: false 2024-04-02T16:22:37.126 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records in 2024-04-02T16:22:37.127 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records out 2024-04-02T16:22:37.127 INFO:teuthology.orchestra.run.smithi050.stderr:70 bytes copied, 2.0949e-05 s, 3.3 MB/s 2024-04-02T16:22:37.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:37.132 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-02T16:22:37.132 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T16:22:37.132 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-02T16:22:37.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:22:37.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.233+0000 7fd13d5c8640 1 Processor -- start 2024-04-02T16:22:37.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.233+0000 7fd13d5c8640 1 -- start start 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd13d5c8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd13815e670 0x7fd1380b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd13d5c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd1380d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd13d5c8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd13809aa40 0x7fd13809aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd13d5c8640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fd13806da20 con 0x7fd13815e670 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd13d5c8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd1380d0b00 con 0x7fd13809aa40 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd13d5c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd1380d0210 con 0x7fd13806cff0 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd13815e670 0x7fd1380b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd13815e670 0x7fd1380b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44256/0 (socket says 172.21.15.50:44256) 2024-04-02T16:22:37.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 -- 172.21.15.50:0/299686516 learned_addr learned my addr 172.21.15.50:0/299686516 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd1367fc640 1 --2- 172.21.15.50:0/299686516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd1380d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd1377fe640 1 --2- 172.21.15.50:0/299686516 >> v2:172.21.15.188:3300/0 conn(0x7fd13809aa40 0x7fd13809aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 -- 172.21.15.50:0/299686516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 msgr2=0x7fd1380d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 --2- 172.21.15.50:0/299686516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd1380d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 -- 172.21.15.50:0/299686516 >> v2:172.21.15.188:3300/0 conn(0x7fd13809aa40 msgr2=0x7fd13809aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 --2- 172.21.15.50:0/299686516 >> v2:172.21.15.188:3300/0 conn(0x7fd13809aa40 0x7fd13809aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.234+0000 7fd136ffd640 1 -- 172.21.15.50:0/299686516 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13806d740 con 0x7fd13815e670 2024-04-02T16:22:37.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.235+0000 7fd136ffd640 1 --2- 172.21.15.50:0/299686516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13815e670 0x7fd1380b4680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd138073160 tx=0x7fd13002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663d7374ffd9eff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:37.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.235+0000 7fd135ffb640 1 -- 172.21.15.50:0/299686516 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130034070 con 0x7fd13815e670 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.235+0000 7fd135ffb640 1 -- 172.21.15.50:0/299686516 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd130030860 con 0x7fd13815e670 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.235+0000 7fd135ffb640 1 -- 172.21.15.50:0/299686516 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130030c30 con 0x7fd13815e670 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 -- 172.21.15.50:0/299686516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13815e670 msgr2=0x7fd1380b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/299686516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13815e670 0x7fd1380b4680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd138073160 tx=0x7fd13002b9a0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 -- 172.21.15.50:0/299686516 shutdown_connections 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/299686516 >> v2:172.21.15.188:3300/0 conn(0x7fd13809aa40 0x7fd13809aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/299686516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd1380d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/299686516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13815e670 0x7fd1380b4680 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 -- 172.21.15.50:0/299686516 >> 172.21.15.50:0/299686516 conn(0x7fd1380b3b90 msgr2=0x7fd1380b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 -- 172.21.15.50:0/299686516 shutdown_connections 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.236+0000 7fd13d5c8640 1 -- 172.21.15.50:0/299686516 wait complete. 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.237+0000 7fd13d5c8640 1 Processor -- start 2024-04-02T16:22:37.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.237+0000 7fd13d5c8640 1 -- start start 2024-04-02T16:22:37.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd13d5c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd13814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:37.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd13d5c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13809aa40 0x7fd1380afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:37.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd13d5c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd13815e670 0x7fd13809edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:37.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd13d5c8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd13806d450 con 0x7fd13809aa40 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd13d5c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1380d0790 con 0x7fd13815e670 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd13d5c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd1380d14d0 con 0x7fd13806cff0 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13809aa40 0x7fd1380afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13809aa40 0x7fd1380afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44272/0 (socket says 172.21.15.50:44272) 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 -- 172.21.15.50:0/1012005611 learned_addr learned my addr 172.21.15.50:0/1012005611 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1377fe640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd13815e670 0x7fd13809edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd136ffd640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd13814d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 -- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 msgr2=0x7fd13814d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd13814d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 -- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd13815e670 msgr2=0x7fd13809edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd13815e670 0x7fd13809edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.238+0000 7fd1367fc640 1 -- 172.21.15.50:0/1012005611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13809d0f0 con 0x7fd13809aa40 2024-04-02T16:22:37.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.239+0000 7fd1367fc640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13809aa40 0x7fd1380afcd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd12000d800 tx=0x7fd12000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.239+0000 7fd12f7fe640 1 -- 172.21.15.50:0/1012005611 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120018020 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.239+0000 7fd12f7fe640 1 -- 172.21.15.50:0/1012005611 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1200096a0 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.239+0000 7fd12f7fe640 1 -- 172.21.15.50:0/1012005611 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120009a70 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.239+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13809d3d0 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.239+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13809dc70 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.240+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13809d5a0 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.240+0000 7fd12f7fe640 1 -- 172.21.15.50:0/1012005611 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd12001b020 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.240+0000 7fd12f7fe640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd11806c8f0 0x7fd11806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.241+0000 7fd12f7fe640 1 -- 172.21.15.50:0/1012005611 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd12008c1b0 con 0x7fd13809aa40 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.241+0000 7fd136ffd640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd11806c8f0 0x7fd11806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:37.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.241+0000 7fd136ffd640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd11806c8f0 0x7fd11806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd130036000 tx=0x7fd13003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:37.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.245+0000 7fd12f7fe640 1 -- 172.21.15.50:0/1012005611 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd120022030 con 0x7fd13809aa40 2024-04-02T16:22:37.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.472+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd1380756f0 con 0x7fd13809aa40 2024-04-02T16:22:37.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.472+0000 7fd12f7fe640 1 -- 172.21.15.50:0/1012005611 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd12005a210 con 0x7fd13809aa40 2024-04-02T16:22:37.473 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd11806c8f0 msgr2=0x7fd11806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd11806c8f0 0x7fd11806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd130036000 tx=0x7fd13003b040 comp rx=0 tx=0).stop 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13809aa40 msgr2=0x7fd1380afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13809aa40 0x7fd1380afcd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd12000d800 tx=0x7fd12000dcc0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 shutdown_connections 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd13815e670 0x7fd13809edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd11806c8f0 0x7fd11806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd13809aa40 0x7fd1380afcd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 --2- 172.21.15.50:0/1012005611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd13806cff0 0x7fd13814d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 >> 172.21.15.50:0/1012005611 conn(0x7fd1380b3b90 msgr2=0x7fd138162860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 shutdown_connections 2024-04-02T16:22:37.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37.474+0000 7fd13d5c8640 1 -- 172.21.15.50:0/1012005611 wait complete. 2024-04-02T16:22:37.485 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T16:22:37.485 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:22:37.485 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-04-02T16:22:37.486 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-02T16:22:37.486 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-02T16:22:37.486 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T16:22:37.486 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T16:22:37.486 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:22:37.486 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-02T16:22:37.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:37.486 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:37.501 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42692sec preferred_lft 42692sec 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:22:37.502 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether a2:a5:44:49:59:06 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::389b:5ff:fe35:52dc/64 scope link tentative 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether a2:a5:44:49:59:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::a0a5:44ff:fe49:5906/64 scope link tentative 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:37.503 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:22:37.566 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-04-02T16:22:37.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:37.568 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-04-02T16:22:37.624 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-04-02T16:22:37.624 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:22:37.624 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T16:22:37.624 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T16:22:37.624 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:22:37.625 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:37.625 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:37.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T16:22:37.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T16:22:37.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T16:22:37.625 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:37.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:37.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:37.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:37.759 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-04-02T16:22:37.773 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T16:22:37.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:37.774 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-04-02T16:22:37.828 INFO:teuthology.orchestra.run.smithi050.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-02T16:22:37.829 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:37.829 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:37.829 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-02T16:22:37.829 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-02T16:22:37.829 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:37.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:37.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:37.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:37.951 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 66:6a:4f:36:fa:16 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-02T16:22:37.977 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-02T16:22:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::646a:4fff:fe36:fa16/64 scope link 2024-04-02T16:22:37.978 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:37.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:37.979 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-02T16:22:38.045 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:38.045 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:38.046 INFO:teuthology.orchestra.run.smithi050.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:22:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 66:6a:4f:36:fa:16 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-02T16:22:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-02T16:22:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::646a:4fff:fe36:fa16/64 scope link 2024-04-02T16:22:38.046 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:38.047 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-02T16:22:38.047 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:38.047 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:38.047 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-02T16:22:38.047 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-02T16:22:38.048 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-02T16:22:38.048 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-02T16:22:38.048 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-02T16:22:38.048 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:38.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:38.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:38.211 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:38.211 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:22:38.211 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.1 up 2024-04-02T16:22:38.211 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.1 master ceph-brx 2024-04-02T16:22:38.211 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:22:38.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:38.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:38.293 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:38.294 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-02T16:22:38.308 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-02T16:22:38.308 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:38.308 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-02T16:22:38.362 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-02T16:22:38.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:38.363 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-02T16:22:38.446 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-02T16:22:38.447 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-02T16:22:38.447 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:22:38.455 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-02T16:22:38.455 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-02T16:22:38.455 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:22:38.455 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:38.455 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-02T16:22:38.460 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=057e6c06-7c98-48a0-9312-9e11e9abe7fe" will pass to kernel 2024-04-02T16:22:38.461 INFO:teuthology.orchestra.run.smithi050.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-02T16:22:38.461 INFO:teuthology.orchestra.run.smithi050.stdout: You just mounted a file system that supports labels which does not 2024-04-02T16:22:38.461 INFO:teuthology.orchestra.run.smithi050.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T16:22:38.461 INFO:teuthology.orchestra.run.smithi050.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T16:22:38.461 INFO:teuthology.orchestra.run.smithi050.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T16:22:38.463 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T16:22:38.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:38.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-02T16:22:38.530 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:38.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:38.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:38.626 INFO:teuthology.orchestra.run.smithi050.stdout:client4592 2024-04-02T16:22:38.686 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:38.686 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:38.702 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:38.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:38.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:38.797 INFO:teuthology.orchestra.run.smithi050.stdout:client4592 2024-04-02T16:22:38.797 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:38.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:38.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:38.854 INFO:teuthology.orchestra.run.smithi050.stdout:client4592 2024-04-02T16:22:38.854 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/057e6c06-7c98-48a0-9312-9e11e9abe7fe.client4592/status') 2024-04-02T16:22:38.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:38.914 INFO:teuthology.orchestra.run.smithi050.stdout:instance: client.4592 (3)172.21.15.50:0/1718659445 2024-04-02T16:22:38.914 INFO:teuthology.orchestra.run.smithi050.stdout:blocklisted: false 2024-04-02T16:22:38.915 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records in 2024-04-02T16:22:38.915 INFO:teuthology.orchestra.run.smithi050.stderr:0+1 records out 2024-04-02T16:22:38.915 INFO:teuthology.orchestra.run.smithi050.stderr:70 bytes copied, 1.9574e-05 s, 3.6 MB/s 2024-04-02T16:22:38.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:38.921 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-02T16:22:38.922 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T16:22:38.922 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-02T16:22:38.922 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:22:39.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.023+0000 7f2b0757b640 1 Processor -- start 2024-04-02T16:22:39.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.023+0000 7f2b0757b640 1 -- start start 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.024+0000 7f2b0757b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 0x7f2b0006ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.024+0000 7f2b0757b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b0006f1d0 con 0x7f2b000d0a80 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.024+0000 7f2b0757b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b0015d3d0 con 0x7f2b000d0620 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.024+0000 7f2b0757b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0015d6b0 con 0x7f2b0006e8b0 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.024+0000 7f2b05b18640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f2b000d0a80 0x7f2b00072320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46568/0 (socket says 172.21.15.50:46568) 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.024+0000 7f2b05b18640 1 -- 172.21.15.50:0/4026406726 learned_addr learned my addr 172.21.15.50:0/4026406726 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05317640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 0x7f2b0006ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 631780822 0 0) 0x7f2b0006f1d0 con 0x7f2b000d0a80 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aec003520 con 0x7f2b000d0a80 2024-04-02T16:22:39.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 566229783 0 0) 0x7f2aec003520 con 0x7f2b000d0a80 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 msgr2=0x7f2b0006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 0x7f2b0006ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 >> v1:172.21.15.188:6789/0 conn(0x7f2b000d0620 legacy=0x7f2b0015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b000cfe20 con 0x7f2b000d0a80 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f2afc0019f0 con 0x7f2b000d0a80 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 >> v1:172.21.15.50:6789/0 conn(0x7f2b000d0a80 legacy=0x7f2b00072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aec0039e0 0x7f2aec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 0x7f2aec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0015d6b0 con 0x7f2aec004d40 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec00ba70 con 0x7f2aec0039e0 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec00bc70 con 0x7f2b0006e8b0 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05b18640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05317640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aec0039e0 0x7f2aec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b04b16640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 0x7f2aec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:39.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05b18640 1 -- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 msgr2=0x7f2aec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05b18640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b0006e8b0 0x7f2aec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05b18640 1 -- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aec0039e0 msgr2=0x7f2aec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05b18640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aec0039e0 0x7f2aec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.025+0000 7f2b05b18640 1 -- 172.21.15.50:0/4026406726 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec003520 con 0x7f2aec004d40 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.026+0000 7f2b05b18640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2b00072290 tx=0x7f2aec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1bf54b6e9bef67 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.026+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00a480 con 0x7f2aec004d40 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.026+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2afc034630 con 0x7f2aec004d40 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.026+0000 7f2af7fff640 1 -- 172.21.15.50:0/4026406726 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc034a40 con 0x7f2aec004d40 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.027+0000 7f2b0757b640 1 -- 172.21.15.50:0/4026406726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 msgr2=0x7f2aec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.027+0000 7f2b0757b640 1 --2- 172.21.15.50:0/4026406726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2b00072290 tx=0x7f2aec005830 comp rx=0 tx=0).stop 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.027+0000 7f2b05b18640 1 -- 172.21.15.50:0/4026406726 reap_dead start 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.027+0000 7f2b0757b640 1 -- 172.21.15.50:0/4026406726 shutdown_connections 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.027+0000 7f2b0757b640 1 -- 172.21.15.50:0/4026406726 >> 172.21.15.50:0/4026406726 conn(0x7f2b000b6540 msgr2=0x7f2b000b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.027+0000 7f2b0757b640 1 -- 172.21.15.50:0/4026406726 shutdown_connections 2024-04-02T16:22:39.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.027+0000 7f2b0757b640 1 -- 172.21.15.50:0/4026406726 wait complete. 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.028+0000 7f2b0757b640 1 Processor -- start 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.028+0000 7f2b0757b640 1 -- start start 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b0757b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2b00141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b0757b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b001f6420 0x7f2b001f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b0757b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b001f68a0 0x7f2b00072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b0757b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00145f80 con 0x7f2aec004d40 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b0757b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00145e00 con 0x7f2b001f68a0 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b0757b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2b000cfa40 con 0x7f2b001f6420 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2b00141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2b00141f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44292/0 (socket says 172.21.15.50:44292) 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 -- 172.21.15.50:0/3001739155 learned_addr learned my addr 172.21.15.50:0/3001739155 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 -- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b001f6420 msgr2=0x7f2b001f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b001f6420 0x7f2b001f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 -- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b001f68a0 msgr2=0x7f2b00072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:39.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b001f68a0 0x7f2b00072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 -- 172.21.15.50:0/3001739155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b00142580 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b05317640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2b00141f70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2af80119d0 tx=0x7f2af8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2af5ffb640 1 -- 172.21.15.50:0/3001739155 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af800fad0 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2af5ffb640 1 -- 172.21.15.50:0/3001739155 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af800fc70 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0013f620 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.029+0000 7f2af5ffb640 1 -- 172.21.15.50:0/3001739155 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8014440 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.030+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0013fbe0 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.031+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b000756f0 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.031+0000 7f2af5ffb640 1 -- 172.21.15.50:0/3001739155 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2af8012040 con 0x7f2aec004d40 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.032+0000 7f2af5ffb640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2ae406c920 0x7f2ae406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:39.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.032+0000 7f2af5ffb640 1 -- 172.21.15.50:0/3001739155 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2af808c4a0 con 0x7f2aec004d40 2024-04-02T16:22:39.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.032+0000 7f2b04b16640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2ae406c920 0x7f2ae406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:39.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.033+0000 7f2b04b16640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2ae406c920 0x7f2ae406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b0013f140 tx=0x7f2af0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:39.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.036+0000 7f2af5ffb640 1 -- 172.21.15.50:0/3001739155 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2af8019030 con 0x7f2aec004d40 2024-04-02T16:22:39.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.258+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b0015f6a0 con 0x7f2aec004d40 2024-04-02T16:22:39.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.259+0000 7f2af5ffb640 1 -- 172.21.15.50:0/3001739155 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f2af8059f40 con 0x7f2aec004d40 2024-04-02T16:22:39.259 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T16:22:39.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2ae406c920 msgr2=0x7f2ae406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:39.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2ae406c920 0x7f2ae406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b0013f140 tx=0x7f2af0009040 comp rx=0 tx=0).stop 2024-04-02T16:22:39.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 msgr2=0x7f2b00141f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:39.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2b00141f70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2af80119d0 tx=0x7f2af8011e90 comp rx=0 tx=0).stop 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 shutdown_connections 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b001f68a0 0x7f2b00072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b001f6420 0x7f2b001f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f2ae406c920 0x7f2ae406ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 --2- 172.21.15.50:0/3001739155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2aec004d40 0x7f2b00141f70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 >> 172.21.15.50:0/3001739155 conn(0x7f2b000b6540 msgr2=0x7f2b0006ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 shutdown_connections 2024-04-02T16:22:39.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:39.261+0000 7f2b0757b640 1 -- 172.21.15.50:0/3001739155 wait complete. 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:22:39.272 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-02T16:22:39.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:39.272 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-04-02T16:22:39.291 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:39.291 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:39.291 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:39.291 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:39.291 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:39.291 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42714sec preferred_lft 42714sec 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2024-04-02T16:22:39.292 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:39.293 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T16:22:39.293 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:39.293 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:39.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2024-04-02T16:22:39.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2024-04-02T16:22:39.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2024-04-02T16:22:39.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T16:22:39.293 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:39.387 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:39.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:39.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:39.500 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:22:39.528 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-04-02T16:22:39.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:39.530 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-04-02T16:22:39.599 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-04-02T16:22:39.599 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:22:39.599 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-02T16:22:39.599 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-02T16:22:39.599 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:22:39.600 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:39.600 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:39.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-02T16:22:39.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-02T16:22:39.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-02T16:22:39.600 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:39.688 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:39.881 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:39.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:39.887 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-02T16:22:39.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:39.902 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-04-02T16:22:39.956 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:39.956 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:39.957 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T16:22:39.957 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-02T16:22:39.957 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:40.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:40.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:40.081 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-02T16:22:40.082 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:40.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:40.221 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:40.227 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:40.227 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:40.228 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-04-02T16:22:40.228 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T16:22:40.228 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:40.277 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:40.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:40.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:40.310 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-02T16:22:40.324 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-02T16:22:40.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:40.325 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-02T16:22:40.379 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-02T16:22:40.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:40.380 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-02T16:22:40.465 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-02T16:22:40.465 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-02T16:22:40.465 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-02T16:22:40.591 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-02T16:22:40.591 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-02T16:22:40.591 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-02T16:22:40.591 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:40.591 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-02T16:22:40.600 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=057e6c06-7c98-48a0-9312-9e11e9abe7fe" will pass to kernel 2024-04-02T16:22:40.601 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-02T16:22:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2024-04-02T16:22:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T16:22:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T16:22:40.601 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T16:22:40.606 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T16:22:40.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:40.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-02T16:22:40.634 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:40.722 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:40.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:40.729 INFO:teuthology.orchestra.run.smithi188.stdout:client4598 2024-04-02T16:22:40.729 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:40.729 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:40.750 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:40.838 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:40.840 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:40.845 INFO:teuthology.orchestra.run.smithi188.stdout:client4598 2024-04-02T16:22:40.846 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:40.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:40.896 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:40.902 INFO:teuthology.orchestra.run.smithi188.stdout:client4598 2024-04-02T16:22:40.902 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/057e6c06-7c98-48a0-9312-9e11e9abe7fe.client4598/status') 2024-04-02T16:22:40.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:40.964 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4598 (3)192.168.0.1:0/2819656000 2024-04-02T16:22:40.964 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2024-04-02T16:22:40.965 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2024-04-02T16:22:40.965 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2024-04-02T16:22:40.965 INFO:teuthology.orchestra.run.smithi188.stderr:69 bytes copied, 3.8474e-05 s, 1.8 MB/s 2024-04-02T16:22:40.965 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:40.971 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-02T16:22:40.971 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T16:22:40.972 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-02T16:22:40.972 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:22:41.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.086+0000 7f5d824d9640 1 Processor -- start 2024-04-02T16:22:41.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.086+0000 7f5d824d9640 1 -- start start 2024-04-02T16:22:41.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.086+0000 7f5d824d9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d7c15e650 0x7f5d7c0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:41.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.086+0000 7f5d824d9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c0b4760 0x7f5d7c0d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:41.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.086+0000 7f5d824d9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d7c0b4c70 con 0x7f5d7c0d2cb0 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.086+0000 7f5d824d9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5d7c0b5880 con 0x7f5d7c15e650 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.086+0000 7f5d824d9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7c0b4f50 con 0x7f5d7c0b4760 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d7bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d7c15e650 0x7f5d7c0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d7b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c0b4760 0x7f5d7c0d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d80a76640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5d7c0d2cb0 0x7f5d7c1622e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:38698/0 (socket says 172.21.15.50:38698) 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d7bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d7c15e650 0x7f5d7c0b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:55056/0 (socket says 172.21.15.50:55056) 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d80a76640 1 -- 172.21.15.50:0/2040303813 learned_addr learned my addr 172.21.15.50:0/2040303813 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d7bfff640 1 -- 172.21.15.50:0/2040303813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c0b4760 msgr2=0x7f5d7c0d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/2040303813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c0b4760 0x7f5d7c0d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d7bfff640 1 -- 172.21.15.50:0/2040303813 >> v1:172.21.15.188:6789/0 conn(0x7f5d7c0d2cb0 legacy=0x7f5d7c1622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:41.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.087+0000 7f5d7bfff640 1 -- 172.21.15.50:0/2040303813 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7c0b5230 con 0x7f5d7c15e650 2024-04-02T16:22:41.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.088+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/2040303813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15e650 0x7f5d7c0b4220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0097a0 tx=0x7f5d6c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36fad7bbc0e84c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:41.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.088+0000 7f5d7affd640 1 -- 172.21.15.50:0/2040303813 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c035020 con 0x7f5d7c15e650 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.088+0000 7f5d7affd640 1 -- 172.21.15.50:0/2040303813 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6c031830 con 0x7f5d7c15e650 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.088+0000 7f5d7affd640 1 -- 172.21.15.50:0/2040303813 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c031c00 con 0x7f5d7c15e650 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 -- 172.21.15.50:0/2040303813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15e650 msgr2=0x7f5d7c0b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 --2- 172.21.15.50:0/2040303813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15e650 0x7f5d7c0b4220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0097a0 tx=0x7f5d6c02b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 -- 172.21.15.50:0/2040303813 shutdown_connections 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 --2- 172.21.15.50:0/2040303813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c0b4760 0x7f5d7c0d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 --2- 172.21.15.50:0/2040303813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c15e650 0x7f5d7c0b4220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 -- 172.21.15.50:0/2040303813 >> 172.21.15.50:0/2040303813 conn(0x7f5d7c09d580 msgr2=0x7f5d7c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 -- 172.21.15.50:0/2040303813 shutdown_connections 2024-04-02T16:22:41.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.089+0000 7f5d824d9640 1 -- 172.21.15.50:0/2040303813 wait complete. 2024-04-02T16:22:41.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.090+0000 7f5d824d9640 1 Processor -- start 2024-04-02T16:22:41.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.090+0000 7f5d824d9640 1 -- start start 2024-04-02T16:22:41.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d824d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c0b4760 0x7f5d7c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:41.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d824d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c0d2cb0 0x7f5d7c1e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:41.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d824d9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c15e650 0x7f5d7c1ea120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d824d9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c15ea30 con 0x7f5d7c0b4760 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d824d9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c0d2270 con 0x7f5d7c0d2cb0 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d824d9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7c15ebb0 con 0x7f5d7c15e650 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c0b4760 0x7f5d7c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c0b4760 0x7f5d7c157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44296/0 (socket says 172.21.15.50:44296) 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d7bfff640 1 -- 172.21.15.50:0/3095740713 learned_addr learned my addr 172.21.15.50:0/3095740713 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d80a76640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c15e650 0x7f5d7c1ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.091+0000 7f5d7b7fe640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c0d2cb0 0x7f5d7c1e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d7bfff640 1 -- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c15e650 msgr2=0x7f5d7c1ea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c15e650 0x7f5d7c1ea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d7bfff640 1 -- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c0d2cb0 msgr2=0x7f5d7c1e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c0d2cb0 0x7f5d7c1e9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d7bfff640 1 -- 172.21.15.50:0/3095740713 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7c0cc530 con 0x7f5d7c0b4760 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d7bfff640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c0b4760 0x7f5d7c157910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0098c0 tx=0x7f5d6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:41.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d78ff9640 1 -- 172.21.15.50:0/3095740713 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c03b980 con 0x7f5d7c0b4760 2024-04-02T16:22:41.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d78ff9640 1 -- 172.21.15.50:0/3095740713 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6c03bb20 con 0x7f5d7c0b4760 2024-04-02T16:22:41.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7c0cc810 con 0x7f5d7c0b4760 2024-04-02T16:22:41.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d78ff9640 1 -- 172.21.15.50:0/3095740713 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0453f0 con 0x7f5d7c0b4760 2024-04-02T16:22:41.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.092+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7c0ccdd0 con 0x7f5d7c0b4760 2024-04-02T16:22:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.093+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d7c0756f0 con 0x7f5d7c0b4760 2024-04-02T16:22:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.093+0000 7f5d78ff9640 1 -- 172.21.15.50:0/3095740713 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d6c04c060 con 0x7f5d7c0b4760 2024-04-02T16:22:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.094+0000 7f5d78ff9640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5d6806c920 0x7f5d6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.094+0000 7f5d78ff9640 1 -- 172.21.15.50:0/3095740713 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f5d6c0ad970 con 0x7f5d7c0b4760 2024-04-02T16:22:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.094+0000 7f5d7b7fe640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5d6806c920 0x7f5d6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:41.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.094+0000 7f5d7b7fe640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5d6806c920 0x7f5d6806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0cf3f0 tx=0x7f5d70009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:41.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.098+0000 7f5d78ff9640 1 -- 172.21.15.50:0/3095740713 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5d6c040070 con 0x7f5d7c0b4760 2024-04-02T16:22:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.324+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5d7c0d1aa0 con 0x7f5d7c0b4760 2024-04-02T16:22:41.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.325+0000 7f5d78ff9640 1 -- 172.21.15.50:0/3095740713 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5d6c031800 con 0x7f5d7c0b4760 2024-04-02T16:22:41.325 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.326+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5d6806c920 msgr2=0x7f5d6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5d6806c920 0x7f5d6806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0cf3f0 tx=0x7f5d70009190 comp rx=0 tx=0).stop 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c0b4760 msgr2=0x7f5d7c157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c0b4760 0x7f5d7c157910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0098c0 tx=0x7f5d6c0046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 shutdown_connections 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d7c15e650 0x7f5d7c1ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d7c0d2cb0 0x7f5d7c1e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5d6806c920 0x7f5d6806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 --2- 172.21.15.50:0/3095740713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d7c0b4760 0x7f5d7c157910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 >> 172.21.15.50:0/3095740713 conn(0x7f5d7c09d580 msgr2=0x7f5d7c0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:41.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 shutdown_connections 2024-04-02T16:22:41.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:41.327+0000 7f5d824d9640 1 -- 172.21.15.50:0/3095740713 wait complete. 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:22:41.338 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-02T16:22:41.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:41.338 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-04-02T16:22:41.355 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42712sec preferred_lft 42712sec 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ae:59:0e:3a:2a:f3 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::545c:94ff:fe39:fd98/64 scope link 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.356 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-02T16:22:41.357 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ae:59:0e:3a:2a:f3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T16:22:41.357 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ac59:eff:fe3a:2af3/64 scope link tentative 2024-04-02T16:22:41.357 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:41.357 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:22:41.422 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-04-02T16:22:41.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:41.424 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-04-02T16:22:41.479 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-04-02T16:22:41.480 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:22:41.480 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-02T16:22:41.480 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-02T16:22:41.480 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:22:41.480 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:41.480 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:41.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-02T16:22:41.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-02T16:22:41.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-02T16:22:41.481 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:41.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:41.609 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:41.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:41.615 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-02T16:22:41.630 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-02T16:22:41.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:41.634 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-04-02T16:22:41.685 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-02T16:22:41.686 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:41.686 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:41.686 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-02T16:22:41.686 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-02T16:22:41.686 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:41.777 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:41.806 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:41.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:41.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-02T16:22:41.838 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether b2:00:f1:6f:6a:0e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b000:f1ff:fe6f:6a0e/64 scope link 2024-04-02T16:22:41.839 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:41.840 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:22:41.907 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether b2:00:f1:6f:6a:0e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-02T16:22:41.908 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-02T16:22:41.908 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.908 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b000:f1ff:fe6f:6a0e/64 scope link 2024-04-02T16:22:41.908 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:22:41.909 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-02T16:22:41.909 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:41.996 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:42.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:42.072 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:22:42.072 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-02T16:22:42.072 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2024-04-02T16:22:42.072 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2024-04-02T16:22:42.072 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-02T16:22:42.119 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:42.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:42.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:42.154 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-02T16:22:42.169 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-02T16:22:42.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:22:42.169 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-02T16:22:42.224 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-02T16:22:42.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:42.224 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-02T16:22:42.308 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-02T16:22:42.308 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-02T16:22:42.308 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-02T16:22:42.317 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-02T16:22:42.317 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-02T16:22:42.317 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-02T16:22:42.317 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:42.317 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-02T16:22:42.327 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=057e6c06-7c98-48a0-9312-9e11e9abe7fe" will pass to kernel 2024-04-02T16:22:42.328 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-02T16:22:42.328 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2024-04-02T16:22:42.328 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T16:22:42.328 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T16:22:42.328 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T16:22:42.329 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T16:22:42.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:42.330 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-02T16:22:42.356 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:42.445 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:42.450 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:42.452 INFO:teuthology.orchestra.run.smithi188.stdout:client4896 2024-04-02T16:22:42.452 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:42.452 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:42.467 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:42.555 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:42.556 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:42.562 INFO:teuthology.orchestra.run.smithi188.stdout:client4896 2024-04-02T16:22:42.562 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:22:42.610 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:22:42.611 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:42.617 INFO:teuthology.orchestra.run.smithi188.stdout:client4896 2024-04-02T16:22:42.617 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/057e6c06-7c98-48a0-9312-9e11e9abe7fe.client4896/status') 2024-04-02T16:22:42.665 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:22:42.678 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4896 (3)172.21.15.188:0/1991022987 2024-04-02T16:22:42.678 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2024-04-02T16:22:42.679 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2024-04-02T16:22:42.679 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2024-04-02T16:22:42.679 INFO:teuthology.orchestra.run.smithi188.stderr:71 bytes copied, 2.1209e-05 s, 3.3 MB/s 2024-04-02T16:22:42.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T16:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:22:42.685 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-02T16:22:42.702 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:42.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.816+0000 7f57483d6640 1 Processor -- start 2024-04-02T16:22:42.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.816+0000 7f57483d6640 1 -- start start 2024-04-02T16:22:42.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f57483d6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f574015c2b0 0x7f57400cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:42.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f57483d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57400b4680 0x7f574009aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:42.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f57483d6640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57400ccce0 con 0x7f574015c690 2024-04-02T16:22:42.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f57483d6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f57400d01b0 con 0x7f574015c2b0 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f57483d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f57400ccf80 con 0x7f57400b4680 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5745971640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f574015c690 0x7f57400cda60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46578/0 (socket says 172.21.15.50:46578) 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5745971640 1 -- 172.21.15.50:0/507866 learned_addr learned my addr 172.21.15.50:0/507866 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5746973640 1 --2- 172.21.15.50:0/507866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57400b4680 0x7f574009aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5746172640 1 --2- 172.21.15.50:0/507866 >> v2:172.21.15.188:3300/0 conn(0x7f574015c2b0 0x7f57400cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5745170640 1 -- 172.21.15.50:0/507866 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 213004311 0 0) 0x7f57400ccce0 con 0x7f574015c690 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5745170640 1 -- 172.21.15.50:0/507866 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f572c003520 con 0x7f574015c690 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5746973640 1 -- 172.21.15.50:0/507866 >> v2:172.21.15.188:3300/0 conn(0x7f574015c2b0 msgr2=0x7f57400cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5746973640 1 --2- 172.21.15.50:0/507866 >> v2:172.21.15.188:3300/0 conn(0x7f574015c2b0 0x7f57400cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5746973640 1 -- 172.21.15.50:0/507866 >> v1:172.21.15.50:6789/0 conn(0x7f574015c690 legacy=0x7f57400cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5746973640 1 -- 172.21.15.50:0/507866 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57400cd260 con 0x7f57400b4680 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.817+0000 7f5746172640 1 --2- 172.21.15.50:0/507866 >> v2:172.21.15.188:3300/0 conn(0x7f574015c2b0 0x7f57400cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.818+0000 7f5746973640 1 --2- 172.21.15.50:0/507866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57400b4680 0x7f574009aa40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5738009d90 tx=0x7f573802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b41fa52242bf78e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.818+0000 7f5745170640 1 -- 172.21.15.50:0/507866 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738039070 con 0x7f57400b4680 2024-04-02T16:22:42.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.818+0000 7f5745170640 1 -- 172.21.15.50:0/507866 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573802cc50 con 0x7f57400b4680 2024-04-02T16:22:42.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.818+0000 7f5745170640 1 -- 172.21.15.50:0/507866 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738034440 con 0x7f57400b4680 2024-04-02T16:22:42.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 -- 172.21.15.50:0/507866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57400b4680 msgr2=0x7f574009aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:42.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 --2- 172.21.15.50:0/507866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57400b4680 0x7f574009aa40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5738009d90 tx=0x7f573802bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:42.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 -- 172.21.15.50:0/507866 shutdown_connections 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 --2- 172.21.15.50:0/507866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57400b4680 0x7f574009aa40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 --2- 172.21.15.50:0/507866 >> v2:172.21.15.188:3300/0 conn(0x7f574015c2b0 0x7f57400cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 -- 172.21.15.50:0/507866 >> 172.21.15.50:0/507866 conn(0x7f57400b3b90 msgr2=0x7f57400b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 -- 172.21.15.50:0/507866 shutdown_connections 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.819+0000 7f57483d6640 1 -- 172.21.15.50:0/507866 wait complete. 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.820+0000 7f57483d6640 1 Processor -- start 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.820+0000 7f57483d6640 1 -- start start 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f57483d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57400b4680 0x7f574015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f57483d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574015c2b0 0x7f57400ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f57483d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574015ed10 0x7f57400ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f57483d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f574009af80 con 0x7f57400b4680 2024-04-02T16:22:42.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f57483d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57400d03e0 con 0x7f574015ed10 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f57483d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f574009b100 con 0x7f574015c2b0 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57400b4680 0x7f574015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57400b4680 0x7f574015f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44312/0 (socket says 172.21.15.50:44312) 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 -- 172.21.15.50:0/1495741017 learned_addr learned my addr 172.21.15.50:0/1495741017 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5745971640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574015c2b0 0x7f57400ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746973640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574015ed10 0x7f57400ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 -- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574015c2b0 msgr2=0x7f57400ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574015c2b0 0x7f57400ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 -- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574015ed10 msgr2=0x7f57400ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574015ed10 0x7f57400ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746172640 1 -- 172.21.15.50:0/1495741017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57401f3370 con 0x7f57400b4680 2024-04-02T16:22:42.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5745971640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574015c2b0 0x7f57400ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:42.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.821+0000 7f5746973640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574015ed10 0x7f57400ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:42.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.822+0000 7f5746172640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57400b4680 0x7f574015f240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f573c00e990 tx=0x7f573c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:42.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.822+0000 7f5736ffd640 1 -- 172.21.15.50:0/1495741017 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c017070 con 0x7f57400b4680 2024-04-02T16:22:42.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.822+0000 7f5736ffd640 1 -- 172.21.15.50:0/1495741017 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c00c9e0 con 0x7f57400b4680 2024-04-02T16:22:42.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.822+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57401f3650 con 0x7f57400b4680 2024-04-02T16:22:42.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.822+0000 7f5736ffd640 1 -- 172.21.15.50:0/1495741017 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c00cdb0 con 0x7f57400b4680 2024-04-02T16:22:42.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.822+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401f3c10 con 0x7f57400b4680 2024-04-02T16:22:42.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.823+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57400756f0 con 0x7f57400b4680 2024-04-02T16:22:42.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.823+0000 7f5736ffd640 1 -- 172.21.15.50:0/1495741017 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f573c01a070 con 0x7f57400b4680 2024-04-02T16:22:42.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.824+0000 7f5736ffd640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f572406c8f0 0x7f572406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:42.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.824+0000 7f5736ffd640 1 -- 172.21.15.50:0/1495741017 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f573c08c230 con 0x7f57400b4680 2024-04-02T16:22:42.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.824+0000 7f5745971640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f572406c8f0 0x7f572406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:42.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.824+0000 7f5745971640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f572406c8f0 0x7f572406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f574009ff90 tx=0x7f5730009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:42.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:42.828+0000 7f5736ffd640 1 -- 172.21.15.50:0/1495741017 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f573c05a240 con 0x7f57400b4680 2024-04-02T16:22:43.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.013+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5740070f80 con 0x7f57400b4680 2024-04-02T16:22:43.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.014+0000 7f5736ffd640 1 -- 172.21.15.50:0/1495741017 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7f573c059b40 con 0x7f57400b4680 2024-04-02T16:22:43.014 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:43.015 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:43.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f572406c8f0 msgr2=0x7f572406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f572406c8f0 0x7f572406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f574009ff90 tx=0x7f5730009040 comp rx=0 tx=0).stop 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57400b4680 msgr2=0x7f574015f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57400b4680 0x7f574015f240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f573c00e990 tx=0x7f573c00ee50 comp rx=0 tx=0).stop 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 shutdown_connections 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574015ed10 0x7f57400ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574015c2b0 0x7f57400ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f572406c8f0 0x7f572406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 --2- 172.21.15.50:0/1495741017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57400b4680 0x7f574015f240 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 >> 172.21.15.50:0/1495741017 conn(0x7f57400b3b90 msgr2=0x7f57400afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 shutdown_connections 2024-04-02T16:22:43.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.016+0000 7f57483d6640 1 -- 172.21.15.50:0/1495741017 wait complete. 2024-04-02T16:22:43.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:43.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:43.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.133+0000 7fd32942a640 1 Processor -- start 2024-04-02T16:22:43.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.133+0000 7fd32942a640 1 -- start start 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd32942a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd324141500 0x7fd324140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd32942a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324140e60 0x7fd324071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd32942a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3240cf450 con 0x7fd32413e120 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd32942a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fd3241412b0 con 0x7fd324141500 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd32942a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240cf730 con 0x7fd324140e60 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd3227fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd324141500 0x7fd324140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd3227fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd324141500 0x7fd324140920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44328/0 (socket says 172.21.15.50:44328) 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.134+0000 7fd3227fc640 1 -- 172.21.15.50:0/3405193217 learned_addr learned my addr 172.21.15.50:0/3405193217 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.135+0000 7fd3237fe640 1 --2- 172.21.15.50:0/3405193217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324140e60 0x7fd324071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.135+0000 7fd3227fc640 1 -- 172.21.15.50:0/3405193217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324140e60 msgr2=0x7fd324071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.135+0000 7fd3227fc640 1 --2- 172.21.15.50:0/3405193217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324140e60 0x7fd324071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.135+0000 7fd3227fc640 1 -- 172.21.15.50:0/3405193217 >> v1:172.21.15.188:6789/0 conn(0x7fd32413e120 legacy=0x7fd3240cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.135+0000 7fd321ffb640 1 -- 172.21.15.50:0/3405193217 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560881292 0 0) 0x7fd3240cf450 con 0x7fd32413e120 2024-04-02T16:22:43.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.135+0000 7fd3227fc640 1 -- 172.21.15.50:0/3405193217 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240cfa10 con 0x7fd324141500 2024-04-02T16:22:43.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.135+0000 7fd3227fc640 1 --2- 172.21.15.50:0/3405193217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324141500 0x7fd324140920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009870 tx=0x7fd30c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ce9b0e1314b7db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd321ffb640 1 -- 172.21.15.50:0/3405193217 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c034070 con 0x7fd324141500 2024-04-02T16:22:43.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd321ffb640 1 -- 172.21.15.50:0/3405193217 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c030840 con 0x7fd324141500 2024-04-02T16:22:43.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd321ffb640 1 -- 172.21.15.50:0/3405193217 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c030c10 con 0x7fd324141500 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 -- 172.21.15.50:0/3405193217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324141500 msgr2=0x7fd324140920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 --2- 172.21.15.50:0/3405193217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324141500 0x7fd324140920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009870 tx=0x7fd30c02b980 comp rx=0 tx=0).stop 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 -- 172.21.15.50:0/3405193217 shutdown_connections 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 --2- 172.21.15.50:0/3405193217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324141500 0x7fd324140920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 --2- 172.21.15.50:0/3405193217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324140e60 0x7fd324071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 -- 172.21.15.50:0/3405193217 >> 172.21.15.50:0/3405193217 conn(0x7fd32406eb90 msgr2=0x7fd32406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 -- 172.21.15.50:0/3405193217 shutdown_connections 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.136+0000 7fd32942a640 1 -- 172.21.15.50:0/3405193217 wait complete. 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.137+0000 7fd32942a640 1 Processor -- start 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd32942a640 1 -- start start 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd32942a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32413e120 0x7fd3240b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd32942a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324140e60 0x7fd3241538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd32942a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324141500 0x7fd3240b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd32942a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd32413e500 con 0x7fd324140e60 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd32942a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd324071400 con 0x7fd32413e120 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd32942a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240cf110 con 0x7fd324141500 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd3227fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324140e60 0x7fd3241538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd3227fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324140e60 0x7fd3241538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44342/0 (socket says 172.21.15.50:44342) 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd3227fc640 1 -- 172.21.15.50:0/1718766099 learned_addr learned my addr 172.21.15.50:0/1718766099 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd322ffd640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32413e120 0x7fd3240b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.138+0000 7fd3237fe640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324141500 0x7fd3240b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd3227fc640 1 -- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324141500 msgr2=0x7fd3240b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd3227fc640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324141500 0x7fd3240b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd3227fc640 1 -- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32413e120 msgr2=0x7fd3240b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd3227fc640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32413e120 0x7fd3240b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd3227fc640 1 -- 172.21.15.50:0/1718766099 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240af920 con 0x7fd324140e60 2024-04-02T16:22:43.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd322ffd640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32413e120 0x7fd3240b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:43.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd3227fc640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324140e60 0x7fd3241538d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009870 tx=0x7fd30c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd31b7fe640 1 -- 172.21.15.50:0/1718766099 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c034040 con 0x7fd324140e60 2024-04-02T16:22:43.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd31b7fe640 1 -- 172.21.15.50:0/1718766099 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c039d90 con 0x7fd324140e60 2024-04-02T16:22:43.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3240afc00 con 0x7fd324140e60 2024-04-02T16:22:43.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd31b7fe640 1 -- 172.21.15.50:0/1718766099 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c043630 con 0x7fd324140e60 2024-04-02T16:22:43.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.139+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3240b01c0 con 0x7fd324140e60 2024-04-02T16:22:43.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.140+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd324075760 con 0x7fd324140e60 2024-04-02T16:22:43.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.141+0000 7fd31b7fe640 1 -- 172.21.15.50:0/1718766099 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd30c04a060 con 0x7fd324140e60 2024-04-02T16:22:43.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.141+0000 7fd31b7fe640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd30406c8f0 0x7fd30406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.141+0000 7fd31b7fe640 1 -- 172.21.15.50:0/1718766099 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fd30c0adea0 con 0x7fd324140e60 2024-04-02T16:22:43.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.141+0000 7fd322ffd640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd30406c8f0 0x7fd30406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.142+0000 7fd322ffd640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd30406c8f0 0x7fd30406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000f60 tx=0x7fd31c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.146+0000 7fd31b7fe640 1 -- 172.21.15.50:0/1718766099 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd30c03e070 con 0x7fd324140e60 2024-04-02T16:22:43.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.330+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3240b5140 con 0x7fd324140e60 2024-04-02T16:22:43.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.331+0000 7fd31b7fe640 1 -- 172.21.15.50:0/1718766099 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fd30c07beb0 con 0x7fd324140e60 2024-04-02T16:22:43.331 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd30406c8f0 msgr2=0x7fd30406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd30406c8f0 0x7fd30406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000f60 tx=0x7fd31c009040 comp rx=0 tx=0).stop 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324140e60 msgr2=0x7fd3241538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324140e60 0x7fd3241538d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009870 tx=0x7fd30c0046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 shutdown_connections 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd324141500 0x7fd3240b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd30406c8f0 0x7fd30406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd324140e60 0x7fd3241538d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 --2- 172.21.15.50:0/1718766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32413e120 0x7fd3240b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 >> 172.21.15.50:0/1718766099 conn(0x7fd32406eb90 msgr2=0x7fd32413c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:43.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 shutdown_connections 2024-04-02T16:22:43.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.333+0000 7fd32942a640 1 -- 172.21.15.50:0/1718766099 wait complete. 2024-04-02T16:22:43.344 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.448+0000 7f13dd760640 1 Processor -- start 2024-04-02T16:22:43.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.448+0000 7f13dd760640 1 -- start start 2024-04-02T16:22:43.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13dd760640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f13d8071020 0x7f13d809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13dd760640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13d809af80 0x7f13d80cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13dd760640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cd9d0 0x7f13d80cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13dd760640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f13d809b3c0 con 0x7f13d8071020 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13dd760640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f13d806cc40 con 0x7f13d809af80 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13dd760640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13d806c960 con 0x7f13d80cd9d0 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f13d8071020 0x7f13d809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f13d8071020 0x7f13d809aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44356/0 (socket says 172.21.15.50:44356) 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2753737061 learned_addr learned my addr 172.21.15.50:0/2753737061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d67fc640 1 --2- 172.21.15.50:0/2753737061 >> v2:172.21.15.188:3300/0 conn(0x7f13d809af80 0x7f13d80cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d77fe640 1 --2- 172.21.15.50:0/2753737061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cd9d0 0x7f13d80cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2753737061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cd9d0 msgr2=0x7f13d80cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 --2- 172.21.15.50:0/2753737061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cd9d0 0x7f13d80cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2753737061 >> v2:172.21.15.188:3300/0 conn(0x7f13d809af80 msgr2=0x7f13d80cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:43.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 --2- 172.21.15.50:0/2753737061 >> v2:172.21.15.188:3300/0 conn(0x7f13d809af80 0x7f13d80cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2753737061 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d80cc6d0 con 0x7f13d8071020 2024-04-02T16:22:43.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.449+0000 7f13d77fe640 1 --2- 172.21.15.50:0/2753737061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cd9d0 0x7f13d80cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:43.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.450+0000 7f13d6ffd640 1 --2- 172.21.15.50:0/2753737061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d809aa40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f13d8068300 tx=0x7f13c802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c823567a6f7b0cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.450+0000 7f13d5ffb640 1 -- 172.21.15.50:0/2753737061 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c802cb00 con 0x7f13d8071020 2024-04-02T16:22:43.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.450+0000 7f13d5ffb640 1 -- 172.21.15.50:0/2753737061 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c802cca0 con 0x7f13d8071020 2024-04-02T16:22:43.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.450+0000 7f13d5ffb640 1 -- 172.21.15.50:0/2753737061 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8034610 con 0x7f13d8071020 2024-04-02T16:22:43.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 -- 172.21.15.50:0/2753737061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 msgr2=0x7f13d809aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 --2- 172.21.15.50:0/2753737061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d809aa40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f13d8068300 tx=0x7f13c802bf60 comp rx=0 tx=0).stop 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 -- 172.21.15.50:0/2753737061 shutdown_connections 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 --2- 172.21.15.50:0/2753737061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cd9d0 0x7f13d80cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 --2- 172.21.15.50:0/2753737061 >> v2:172.21.15.188:3300/0 conn(0x7f13d809af80 0x7f13d80cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 --2- 172.21.15.50:0/2753737061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d809aa40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 -- 172.21.15.50:0/2753737061 >> 172.21.15.50:0/2753737061 conn(0x7f13d80b3ad0 msgr2=0x7f13d80b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 -- 172.21.15.50:0/2753737061 shutdown_connections 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.451+0000 7f13dd760640 1 -- 172.21.15.50:0/2753737061 wait complete. 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.452+0000 7f13dd760640 1 Processor -- start 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.452+0000 7f13dd760640 1 -- start start 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13dd760640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d80c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13dd760640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d809af80 0x7f13d80b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13dd760640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cd9d0 0x7f13d8153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13dd760640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80ccf70 con 0x7f13d8071020 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13dd760640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80ccc90 con 0x7f13d80cd9d0 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13dd760640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13d806db50 con 0x7f13d809af80 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d80c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d80c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44368/0 (socket says 172.21.15.50:44368) 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2517852187 learned_addr learned my addr 172.21.15.50:0/2517852187 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d67fc640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d809af80 0x7f13d80b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d77fe640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cd9d0 0x7f13d8153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d809af80 msgr2=0x7f13d80b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d809af80 0x7f13d80b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cd9d0 msgr2=0x7f13d8153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cd9d0 0x7f13d8153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d6ffd640 1 -- 172.21.15.50:0/2517852187 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d80c7a20 con 0x7f13d8071020 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d67fc640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d809af80 0x7f13d80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.453+0000 7f13d77fe640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cd9d0 0x7f13d8153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.454+0000 7f13d6ffd640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d80c8a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009f40 tx=0x7f13c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.454+0000 7f13cf7fe640 1 -- 172.21.15.50:0/2517852187 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8043020 con 0x7f13d8071020 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.454+0000 7f13cf7fe640 1 -- 172.21.15.50:0/2517852187 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c802ce50 con 0x7f13d8071020 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.454+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d80c7cc0 con 0x7f13d8071020 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.454+0000 7f13cf7fe640 1 -- 172.21.15.50:0/2517852187 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8009510 con 0x7f13d8071020 2024-04-02T16:22:43.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.454+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d80c8240 con 0x7f13d8071020 2024-04-02T16:22:43.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.455+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d8075760 con 0x7f13d8071020 2024-04-02T16:22:43.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.455+0000 7f13cf7fe640 1 -- 172.21.15.50:0/2517852187 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13c8046020 con 0x7f13d8071020 2024-04-02T16:22:43.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.456+0000 7f13cf7fe640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f13b806c8f0 0x7f13b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.456+0000 7f13cf7fe640 1 -- 172.21.15.50:0/2517852187 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f13c80ae600 con 0x7f13d8071020 2024-04-02T16:22:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.456+0000 7f13d67fc640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f13b806c8f0 0x7f13b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.456+0000 7f13d67fc640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f13b806c8f0 0x7f13b806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f13d806d310 tx=0x7f13c0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.460+0000 7f13cf7fe640 1 -- 172.21.15.50:0/2517852187 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f13c804d030 con 0x7f13d8071020 2024-04-02T16:22:43.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.643+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13d809ad60 con 0x7f13d8071020 2024-04-02T16:22:43.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.643+0000 7f13cf7fe640 1 -- 172.21.15.50:0/2517852187 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f13c807bfc0 con 0x7f13d8071020 2024-04-02T16:22:43.643 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f13b806c8f0 msgr2=0x7f13b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f13b806c8f0 0x7f13b806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f13d806d310 tx=0x7f13c0008040 comp rx=0 tx=0).stop 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 msgr2=0x7f13d80c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d80c8a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009f40 tx=0x7f13c8004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 shutdown_connections 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cd9d0 0x7f13d8153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d809af80 0x7f13d80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f13b806c8f0 0x7f13b806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 --2- 172.21.15.50:0/2517852187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13d8071020 0x7f13d80c8a40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 >> 172.21.15.50:0/2517852187 conn(0x7f13d80b3ad0 msgr2=0x7f13d80cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 shutdown_connections 2024-04-02T16:22:43.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.645+0000 7f13dd760640 1 -- 172.21.15.50:0/2517852187 wait complete. 2024-04-02T16:22:43.656 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:43.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.761+0000 7ff7de4f2640 1 Processor -- start 2024-04-02T16:22:43.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.761+0000 7ff7de4f2640 1 -- start start 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.761+0000 7ff7de4f2640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff7d80c7e30 0x7ff7d80d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.761+0000 7ff7de4f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e7b0 0x7ff7d80d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.761+0000 7ff7de4f2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d80d2f30 con 0x7ff7d815e3d0 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.761+0000 7ff7de4f2640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d80cf000 con 0x7ff7d80c7e30 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.761+0000 7ff7de4f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d80cf2e0 con 0x7ff7d815e7b0 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff7d80c7e30 0x7ff7d80d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d7fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff7d815e3d0 0x7ff7d80d4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46580/0 (socket says 172.21.15.50:46580) 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d7fff640 1 -- 172.21.15.50:0/2603610259 learned_addr learned my addr 172.21.15.50:0/2603610259 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7dca8f640 1 --2- 172.21.15.50:0/2603610259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e7b0 0x7ff7d80d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d77fe640 1 -- 172.21.15.50:0/2603610259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e7b0 msgr2=0x7ff7d80d26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d77fe640 1 --2- 172.21.15.50:0/2603610259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e7b0 0x7ff7d80d26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d77fe640 1 -- 172.21.15.50:0/2603610259 >> v1:172.21.15.50:6789/0 conn(0x7ff7d815e3d0 legacy=0x7ff7d80d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d77fe640 1 -- 172.21.15.50:0/2603610259 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d80cf5c0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.762+0000 7ff7d6ffd640 1 -- 172.21.15.50:0/2603610259 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4154023139 0 0) 0x7ff7d80d2f30 con 0x7ff7d815e3d0 2024-04-02T16:22:43.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7d77fe640 1 --2- 172.21.15.50:0/2603610259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d80d2170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0097a0 tx=0x7ff7cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5a08a55075276 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7d6ffd640 1 -- 172.21.15.50:0/2603610259 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc035020 con 0x7ff7d80c7e30 2024-04-02T16:22:43.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7d6ffd640 1 -- 172.21.15.50:0/2603610259 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7cc031830 con 0x7ff7d80c7e30 2024-04-02T16:22:43.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7d6ffd640 1 -- 172.21.15.50:0/2603610259 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc031c00 con 0x7ff7d80c7e30 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 -- 172.21.15.50:0/2603610259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 msgr2=0x7ff7d80d2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/2603610259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d80d2170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0097a0 tx=0x7ff7cc02b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 -- 172.21.15.50:0/2603610259 shutdown_connections 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/2603610259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e7b0 0x7ff7d80d26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/2603610259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d80d2170 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 -- 172.21.15.50:0/2603610259 >> 172.21.15.50:0/2603610259 conn(0x7ff7d808dc90 msgr2=0x7ff7d809d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 -- 172.21.15.50:0/2603610259 shutdown_connections 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.763+0000 7ff7de4f2640 1 -- 172.21.15.50:0/2603610259 wait complete. 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.764+0000 7ff7de4f2640 1 Processor -- start 2024-04-02T16:22:43.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7de4f2640 1 -- start start 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7de4f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7de4f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e3d0 0x7ff7d80cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7de4f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7d815e7b0 0x7ff7d80b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7de4f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d80d3390 con 0x7ff7d80c7e30 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7de4f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d80d3210 con 0x7ff7d815e7b0 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7de4f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d80d4740 con 0x7ff7d815e3d0 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.765+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d815b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44394/0 (socket says 172.21.15.50:44394) 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 -- 172.21.15.50:0/93446472 learned_addr learned my addr 172.21.15.50:0/93446472 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7dca8f640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7d815e7b0 0x7ff7d80b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d77fe640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e3d0 0x7ff7d80cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 -- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e3d0 msgr2=0x7ff7d80cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e3d0 0x7ff7d80cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 -- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7d815e7b0 msgr2=0x7ff7d80b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7d815e7b0 0x7ff7d80b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 -- 172.21.15.50:0/93446472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d80b7410 con 0x7ff7d80c7e30 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d7fff640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d815b170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800c4a0 tx=0x7ff7c800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d4ff9640 1 -- 172.21.15.50:0/93446472 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8017070 con 0x7ff7d80c7e30 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d4ff9640 1 -- 172.21.15.50:0/93446472 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c8005ce0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d80b76f0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7d4ff9640 1 -- 172.21.15.50:0/93446472 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80072b0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.766+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d80b7cb0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.767+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d80d3aa0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.768+0000 7ff7d4ff9640 1 -- 172.21.15.50:0/93446472 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7c801a070 con 0x7ff7d80c7e30 2024-04-02T16:22:43.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.768+0000 7ff7d4ff9640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff7c406c8f0 0x7ff7c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:43.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.768+0000 7ff7d4ff9640 1 -- 172.21.15.50:0/93446472 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff7c808b2a0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.768+0000 7ff7d77fe640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff7c406c8f0 0x7ff7c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:43.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.769+0000 7ff7d77fe640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff7c406c8f0 0x7ff7c406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc036000 tx=0x7ff7cc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:43.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.772+0000 7ff7d4ff9640 1 -- 172.21.15.50:0/93446472 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff7d80d3aa0 con 0x7ff7d80c7e30 2024-04-02T16:22:43.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.956+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7d8075760 con 0x7ff7d80c7e30 2024-04-02T16:22:43.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.956+0000 7ff7d4ff9640 1 -- 172.21.15.50:0/93446472 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ff7c8059340 con 0x7ff7d80c7e30 2024-04-02T16:22:43.957 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff7c406c8f0 msgr2=0x7ff7c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff7c406c8f0 0x7ff7c406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc036000 tx=0x7ff7cc008030 comp rx=0 tx=0).stop 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 msgr2=0x7ff7d815b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d815b170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800c4a0 tx=0x7ff7c800c960 comp rx=0 tx=0).stop 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 shutdown_connections 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7d815e7b0 0x7ff7d80b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7d815e3d0 0x7ff7d80cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff7c406c8f0 0x7ff7c406edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 --2- 172.21.15.50:0/93446472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d80c7e30 0x7ff7d815b170 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 >> 172.21.15.50:0/93446472 conn(0x7ff7d808dc90 msgr2=0x7ff7d80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 shutdown_connections 2024-04-02T16:22:43.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:43.958+0000 7ff7de4f2640 1 -- 172.21.15.50:0/93446472 wait complete. 2024-04-02T16:22:43.969 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:44.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.073+0000 7f9f4519f640 1 Processor -- start 2024-04-02T16:22:44.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.073+0000 7f9f4519f640 1 -- start start 2024-04-02T16:22:44.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f4519f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f400b4c30 0x7f9f4015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f4519f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f4015cfa0 0x7f9f400cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f4519f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f400b3bf0 con 0x7f9f400cc770 2024-04-02T16:22:44.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f4519f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f9f4015a1f0 con 0x7f9f400b4c30 2024-04-02T16:22:44.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f4519f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9f400b3e70 con 0x7f9f4015cfa0 2024-04-02T16:22:44.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f3f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f400b4c30 0x7f9f4015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f3f7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9f400b4c30 0x7f9f4015ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44396/0 (socket says 172.21.15.50:44396) 2024-04-02T16:22:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.074+0000 7f9f3f7fe640 1 -- 172.21.15.50:0/478944333 learned_addr learned my addr 172.21.15.50:0/478944333 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.075+0000 7f9f4499e640 1 --2- 172.21.15.50:0/478944333 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015cfa0 0x7f9f400cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.075+0000 7f9f3f7fe640 1 -- 172.21.15.50:0/478944333 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015cfa0 msgr2=0x7f9f400cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.075+0000 7f9f3f7fe640 1 --2- 172.21.15.50:0/478944333 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015cfa0 0x7f9f400cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.075+0000 7f9f3f7fe640 1 -- 172.21.15.50:0/478944333 >> v1:172.21.15.188:6789/0 conn(0x7f9f400cc770 legacy=0x7f9f400ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:44.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.075+0000 7f9f3f7fe640 1 -- 172.21.15.50:0/478944333 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f400b4150 con 0x7f9f400b4c30 2024-04-02T16:22:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.075+0000 7f9f3f7fe640 1 --2- 172.21.15.50:0/478944333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4015ca60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9f400b2a10 tx=0x7f9f3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da80d5a9969731c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f3effd640 1 -- 172.21.15.50:0/478944333 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34035020 con 0x7f9f400b4c30 2024-04-02T16:22:44.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f3effd640 1 -- 172.21.15.50:0/478944333 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f34031870 con 0x7f9f400b4c30 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f3effd640 1 -- 172.21.15.50:0/478944333 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34031c40 con 0x7f9f400b4c30 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 -- 172.21.15.50:0/478944333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 msgr2=0x7f9f4015ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 --2- 172.21.15.50:0/478944333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4015ca60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9f400b2a10 tx=0x7f9f3402b910 comp rx=0 tx=0).stop 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 -- 172.21.15.50:0/478944333 shutdown_connections 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 --2- 172.21.15.50:0/478944333 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015cfa0 0x7f9f400cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 --2- 172.21.15.50:0/478944333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4015ca60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 -- 172.21.15.50:0/478944333 >> 172.21.15.50:0/478944333 conn(0x7f9f400b20c0 msgr2=0x7f9f400cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 -- 172.21.15.50:0/478944333 shutdown_connections 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.076+0000 7f9f4519f640 1 -- 172.21.15.50:0/478944333 wait complete. 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.077+0000 7f9f4519f640 1 Processor -- start 2024-04-02T16:22:44.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f4519f640 1 -- start start 2024-04-02T16:22:44.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f4519f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4014b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f4519f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400cc770 0x7f9f400adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f4519f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015cfa0 0x7f9f400ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f4519f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4015d410 con 0x7f9f400b4c30 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f4519f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f400b4710 con 0x7f9f4015cfa0 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f4519f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f400ce140 con 0x7f9f400cc770 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f3ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4014b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f3ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4014b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44398/0 (socket says 172.21.15.50:44398) 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.078+0000 7f9f3ffff640 1 -- 172.21.15.50:0/3159970172 learned_addr learned my addr 172.21.15.50:0/3159970172 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f4499e640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015cfa0 0x7f9f400ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3f7fe640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400cc770 0x7f9f400adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3ffff640 1 -- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400cc770 msgr2=0x7f9f400adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3ffff640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400cc770 0x7f9f400adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3ffff640 1 -- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015cfa0 msgr2=0x7f9f400ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3ffff640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015cfa0 0x7f9f400ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3ffff640 1 -- 172.21.15.50:0/3159970172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f401f5b10 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f4499e640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015cfa0 0x7f9f400ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3ffff640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4014b900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c430 tx=0x7f9f3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3cff9640 1 -- 172.21.15.50:0/3159970172 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30011790 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3cff9640 1 -- 172.21.15.50:0/3159970172 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f30017070 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f401f5df0 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f3cff9640 1 -- 172.21.15.50:0/3159970172 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30011bb0 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.079+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f401f63b0 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.080+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4004f940 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.081+0000 7f9f3cff9640 1 -- 172.21.15.50:0/3159970172 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f3001c460 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.081+0000 7f9f3cff9640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f2c06c920 0x7f9f2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.081+0000 7f9f3cff9640 1 -- 172.21.15.50:0/3159970172 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f9f3008bd60 con 0x7f9f400b4c30 2024-04-02T16:22:44.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.081+0000 7f9f3f7fe640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f2c06c920 0x7f9f2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.082+0000 7f9f3f7fe640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f2c06c920 0x7f9f2c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f34036000 tx=0x7f9f3403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.085+0000 7f9f3cff9640 1 -- 172.21.15.50:0/3159970172 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9f30059e00 con 0x7f9f400b4c30 2024-04-02T16:22:44.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.268+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9f400cf510 con 0x7f9f400b4c30 2024-04-02T16:22:44.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.268+0000 7f9f3cff9640 1 -- 172.21.15.50:0/3159970172 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f9f30059700 con 0x7f9f400b4c30 2024-04-02T16:22:44.269 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f2c06c920 msgr2=0x7f9f2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f2c06c920 0x7f9f2c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f34036000 tx=0x7f9f3403c040 comp rx=0 tx=0).stop 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 msgr2=0x7f9f4014b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4014b900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c430 tx=0x7f9f3000c8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 shutdown_connections 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015cfa0 0x7f9f400ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400cc770 0x7f9f400adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f2c06c920 0x7f9f2c06ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 --2- 172.21.15.50:0/3159970172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f400b4c30 0x7f9f4014b900 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 >> 172.21.15.50:0/3159970172 conn(0x7f9f400b20c0 msgr2=0x7f9f4006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 shutdown_connections 2024-04-02T16:22:44.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.270+0000 7f9f4519f640 1 -- 172.21.15.50:0/3159970172 wait complete. 2024-04-02T16:22:44.282 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:44.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.383+0000 7f5215e9a640 1 Processor -- start 2024-04-02T16:22:44.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.383+0000 7f5215e9a640 1 -- start start 2024-04-02T16:22:44.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f5215e9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b3b40 0x7f52100b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f5215e9a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f52100b4a50 0x7f5210142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f5215e9a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52100b5400 con 0x7f52100b1bc0 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f5215e9a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f52100b4ee0 con 0x7f52100b4a50 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f5215e9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52100b5120 con 0x7f52100b3b40 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f52100b4a50 0x7f5210142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f52100b4a50 0x7f5210142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44404/0 (socket says 172.21.15.50:44404) 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520ffff640 1 -- 172.21.15.50:0/2377152520 learned_addr learned my addr 172.21.15.50:0/2377152520 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520effd640 1 --2- 172.21.15.50:0/2377152520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b3b40 0x7f52100b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520ffff640 1 -- 172.21.15.50:0/2377152520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b3b40 msgr2=0x7f52100b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520ffff640 1 --2- 172.21.15.50:0/2377152520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b3b40 0x7f52100b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520ffff640 1 -- 172.21.15.50:0/2377152520 >> v1:172.21.15.188:6789/0 conn(0x7f52100b1bc0 legacy=0x7f52100b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:44.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.384+0000 7f520ffff640 1 -- 172.21.15.50:0/2377152520 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52100b3060 con 0x7f52100b4a50 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.385+0000 7f520ffff640 1 --2- 172.21.15.50:0/2377152520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210142fb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5208009e50 tx=0x7f520802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4212abfaa769fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.385+0000 7f520e7fc640 1 -- 172.21.15.50:0/2377152520 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520802c9e0 con 0x7f52100b4a50 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.385+0000 7f520e7fc640 1 -- 172.21.15.50:0/2377152520 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520802cb80 con 0x7f52100b4a50 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.385+0000 7f520e7fc640 1 -- 172.21.15.50:0/2377152520 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080343f0 con 0x7f52100b4a50 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.385+0000 7f5215e9a640 1 -- 172.21.15.50:0/2377152520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 msgr2=0x7f5210142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.385+0000 7f5215e9a640 1 --2- 172.21.15.50:0/2377152520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210142fb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5208009e50 tx=0x7f520802be40 comp rx=0 tx=0).stop 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.386+0000 7f5215e9a640 1 -- 172.21.15.50:0/2377152520 shutdown_connections 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.386+0000 7f5215e9a640 1 --2- 172.21.15.50:0/2377152520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210142fb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.386+0000 7f5215e9a640 1 --2- 172.21.15.50:0/2377152520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b3b40 0x7f52100b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.386+0000 7f5215e9a640 1 -- 172.21.15.50:0/2377152520 >> 172.21.15.50:0/2377152520 conn(0x7f521006ebf0 msgr2=0x7f521006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.386+0000 7f5215e9a640 1 -- 172.21.15.50:0/2377152520 shutdown_connections 2024-04-02T16:22:44.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.386+0000 7f5215e9a640 1 -- 172.21.15.50:0/2377152520 wait complete. 2024-04-02T16:22:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.387+0000 7f5215e9a640 1 Processor -- start 2024-04-02T16:22:44.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.387+0000 7f5215e9a640 1 -- start start 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f5215e9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b1bc0 0x7f5210159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f5215e9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52100b3b40 0x7f52100aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f5215e9a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f5215e9a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5210142b00 con 0x7f52100b4a50 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f5215e9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52100b3620 con 0x7f52100b3b40 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f5215e9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5210142c80 con 0x7f52100b1bc0 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44420/0 (socket says 172.21.15.50:44420) 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 -- 172.21.15.50:0/1699030903 learned_addr learned my addr 172.21.15.50:0/1699030903 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:44.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 -- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b1bc0 msgr2=0x7f5210159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520f7fe640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b1bc0 0x7f5210159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520effd640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52100b3b40 0x7f52100aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b1bc0 0x7f5210159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 -- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52100b3b40 msgr2=0x7f52100aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52100b3b40 0x7f52100aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 -- 172.21.15.50:0/1699030903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52100ad8b0 con 0x7f52100b4a50 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520f7fe640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b1bc0 0x7f5210159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520effd640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52100b3b40 0x7f52100aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f520ffff640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210071c70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5208036040 tx=0x7f5208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f51f7fff640 1 -- 172.21.15.50:0/1699030903 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208043020 con 0x7f52100b4a50 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f51f7fff640 1 -- 172.21.15.50:0/1699030903 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520803e450 con 0x7f52100b4a50 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f51f7fff640 1 -- 172.21.15.50:0/1699030903 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208009510 con 0x7f52100b4a50 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.388+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52100adb90 con 0x7f52100b4a50 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.389+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52100ae150 con 0x7f52100b4a50 2024-04-02T16:22:44.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.389+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52100756f0 con 0x7f52100b4a50 2024-04-02T16:22:44.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.390+0000 7f51f7fff640 1 -- 172.21.15.50:0/1699030903 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5208046020 con 0x7f52100b4a50 2024-04-02T16:22:44.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.390+0000 7f51f7fff640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f51fc06c8f0 0x7f51fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.390+0000 7f51f7fff640 1 -- 172.21.15.50:0/1699030903 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f52080ae7b0 con 0x7f52100b4a50 2024-04-02T16:22:44.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.390+0000 7f520f7fe640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f51fc06c8f0 0x7f51fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.391+0000 7f520f7fe640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f51fc06c8f0 0x7f51fc06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f520000a830 tx=0x7f5200008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.395+0000 7f51f7fff640 1 -- 172.21.15.50:0/1699030903 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f520804d030 con 0x7f52100b4a50 2024-04-02T16:22:44.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.581+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f521006c280 con 0x7f52100b4a50 2024-04-02T16:22:44.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.582+0000 7f51f7fff640 1 -- 172.21.15.50:0/1699030903 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7f520807c7b0 con 0x7f52100b4a50 2024-04-02T16:22:44.583 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:44.583 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f51fc06c8f0 msgr2=0x7f51fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f51fc06c8f0 0x7f51fc06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f520000a830 tx=0x7f5200008040 comp rx=0 tx=0).stop 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 msgr2=0x7f5210071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210071c70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5208036040 tx=0x7f5208004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 shutdown_connections 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f51fc06c8f0 0x7f51fc06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52100b4a50 0x7f5210071c70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52100b3b40 0x7f52100aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 --2- 172.21.15.50:0/1699030903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100b1bc0 0x7f5210159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 >> 172.21.15.50:0/1699030903 conn(0x7f521006ebf0 msgr2=0x7f52101606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 shutdown_connections 2024-04-02T16:22:44.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.584+0000 7f5215e9a640 1 -- 172.21.15.50:0/1699030903 wait complete. 2024-04-02T16:22:44.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:44.596 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:44.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.701+0000 7f0af8b04640 1 Processor -- start 2024-04-02T16:22:44.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.701+0000 7f0af8b04640 1 -- start start 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.701+0000 7f0af8b04640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af415e3d0 0x7f0af40d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.701+0000 7f0af8b04640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0af415e7b0 0x7f0af40d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.701+0000 7f0af8b04640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af40d2f30 con 0x7f0af40c7e30 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.701+0000 7f0af8b04640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f0af40cf000 con 0x7f0af415e7b0 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.701+0000 7f0af8b04640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0af40cf2e0 con 0x7f0af415e3d0 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af2d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0af415e7b0 0x7f0af40d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af2d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0af415e7b0 0x7f0af40d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44428/0 (socket says 172.21.15.50:44428) 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1164258822 learned_addr learned my addr 172.21.15.50:0/1164258822 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af259c640 1 --2- 172.21.15.50:0/1164258822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af415e3d0 0x7f0af40d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1164258822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af415e3d0 msgr2=0x7f0af40d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:44.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af2d9d640 1 --2- 172.21.15.50:0/1164258822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af415e3d0 0x7f0af40d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1164258822 >> v1:172.21.15.188:6789/0 conn(0x7f0af40c7e30 legacy=0x7f0af40d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1164258822 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af40cf5c0 con 0x7f0af415e7b0 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.702+0000 7f0af259c640 1 --2- 172.21.15.50:0/1164258822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af415e3d0 0x7f0af40d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af2d9d640 1 --2- 172.21.15.50:0/1164258822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af40d2880 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e50 tx=0x7f0aec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f94a93e13b0b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af1d9b640 1 -- 172.21.15.50:0/1164258822 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec02ca70 con 0x7f0af415e7b0 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af1d9b640 1 -- 172.21.15.50:0/1164258822 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec02cc10 con 0x7f0af415e7b0 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af1d9b640 1 -- 172.21.15.50:0/1164258822 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0343f0 con 0x7f0af415e7b0 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 -- 172.21.15.50:0/1164258822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 msgr2=0x7f0af40d2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1164258822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af40d2880 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e50 tx=0x7f0aec02bed0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 -- 172.21.15.50:0/1164258822 shutdown_connections 2024-04-02T16:22:44.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1164258822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af40d2880 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1164258822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af415e3d0 0x7f0af40d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 -- 172.21.15.50:0/1164258822 >> 172.21.15.50:0/1164258822 conn(0x7f0af408dc90 msgr2=0x7f0af409d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:44.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 -- 172.21.15.50:0/1164258822 shutdown_connections 2024-04-02T16:22:44.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.703+0000 7f0af8b04640 1 -- 172.21.15.50:0/1164258822 wait complete. 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 Processor -- start 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 -- start start 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af40c7e30 0x7f0af41ea660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af415e3d0 0x7f0af41597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af41605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40d46f0 con 0x7f0af415e7b0 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40d4570 con 0x7f0af415e3d0 2024-04-02T16:22:44.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af8b04640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0af40d4870 con 0x7f0af40c7e30 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af2d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af41605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af2d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af41605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44430/0 (socket says 172.21.15.50:44430) 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.705+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1583222292 learned_addr learned my addr 172.21.15.50:0/1583222292 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af259c640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af40c7e30 0x7f0af41ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0aebfff640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af415e3d0 0x7f0af41597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af40c7e30 msgr2=0x7f0af41ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af2d9d640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af40c7e30 0x7f0af41ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af415e3d0 msgr2=0x7f0af41597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af2d9d640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af415e3d0 0x7f0af41597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af2d9d640 1 -- 172.21.15.50:0/1583222292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af4160b30 con 0x7f0af415e7b0 2024-04-02T16:22:44.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af2d9d640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af41605f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e50 tx=0x7f0aec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0aeaffd640 1 -- 172.21.15.50:0/1583222292 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec043020 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0aeaffd640 1 -- 172.21.15.50:0/1583222292 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec03e450 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af41e95c0 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0aeaffd640 1 -- 172.21.15.50:0/1583222292 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec03e820 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.706+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af41e9b80 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.707+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af4075760 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.708+0000 7f0aeaffd640 1 -- 172.21.15.50:0/1583222292 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0aec046020 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.708+0000 7f0aeaffd640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.708+0000 7f0aeaffd640 1 -- 172.21.15.50:0/1583222292 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f0aec0ae860 con 0x7f0af415e7b0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.708+0000 7f0af259c640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:44.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.709+0000 7f0af259c640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0ad406c8f0 0x7f0ad406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400a990 tx=0x7f0ae4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:44.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.712+0000 7f0aeaffd640 1 -- 172.21.15.50:0/1583222292 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0aec04d030 con 0x7f0af415e7b0 2024-04-02T16:22:44.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.895+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0af40d51d0 con 0x7f0af415e7b0 2024-04-02T16:22:44.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.895+0000 7f0aeaffd640 1 -- 172.21.15.50:0/1583222292 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f0aec07c870 con 0x7f0af415e7b0 2024-04-02T16:22:44.899 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:44.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0ad406c8f0 msgr2=0x7f0ad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0ad406c8f0 0x7f0ad406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400a990 tx=0x7f0ae4008040 comp rx=0 tx=0).stop 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 msgr2=0x7f0af41605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af41605f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e50 tx=0x7f0aec004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 shutdown_connections 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0af415e7b0 0x7f0af41605f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af415e3d0 0x7f0af41597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 --2- 172.21.15.50:0/1583222292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af40c7e30 0x7f0af41ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 >> 172.21.15.50:0/1583222292 conn(0x7f0af408dc90 msgr2=0x7f0af40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 shutdown_connections 2024-04-02T16:22:44.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:44.897+0000 7f0af8b04640 1 -- 172.21.15.50:0/1583222292 wait complete. 2024-04-02T16:22:44.908 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.013+0000 7f3928422640 1 Processor -- start 2024-04-02T16:22:45.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.013+0000 7f3928422640 1 -- start start 2024-04-02T16:22:45.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.013+0000 7f3928422640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f392015e3d0 0x7f39200d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.013+0000 7f3928422640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f39200c7e30 0x7f39200d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.013+0000 7f3928422640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39200d2f30 con 0x7f392015e7b0 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.013+0000 7f3928422640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f39200cf000 con 0x7f39200c7e30 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.013+0000 7f3928422640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f39200cf2e0 con 0x7f392015e3d0 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39259bd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f39200c7e30 0x7f39200d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39269bf640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f392015e7b0 0x7f39200d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:46588/0 (socket says 172.21.15.50:46588) 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39269bf640 1 -- 172.21.15.50:0/1661001771 learned_addr learned my addr 172.21.15.50:0/1661001771 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39251bc640 1 -- 172.21.15.50:0/1661001771 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899993872 0 0) 0x7f39200d2f30 con 0x7f392015e7b0 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39251bc640 1 -- 172.21.15.50:0/1661001771 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f390c003520 con 0x7f392015e7b0 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39261be640 1 --2- 172.21.15.50:0/1661001771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f392015e3d0 0x7f39200d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39259bd640 1 -- 172.21.15.50:0/1661001771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f392015e3d0 msgr2=0x7f39200d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39259bd640 1 --2- 172.21.15.50:0/1661001771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f392015e3d0 0x7f39200d4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39259bd640 1 -- 172.21.15.50:0/1661001771 >> v1:172.21.15.50:6789/0 conn(0x7f392015e7b0 legacy=0x7f39200d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.014+0000 7f39259bd640 1 -- 172.21.15.50:0/1661001771 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39200cf5c0 con 0x7f39200c7e30 2024-04-02T16:22:45.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f39259bd640 1 --2- 172.21.15.50:0/1661001771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39200c7e30 0x7f39200d4570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39100097a0 tx=0x7f391002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804487b42542a525 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f39251bc640 1 -- 172.21.15.50:0/1661001771 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910035020 con 0x7f39200c7e30 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f39251bc640 1 -- 172.21.15.50:0/1661001771 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910031830 con 0x7f39200c7e30 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f39251bc640 1 -- 172.21.15.50:0/1661001771 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910031c00 con 0x7f39200c7e30 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f3928422640 1 -- 172.21.15.50:0/1661001771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39200c7e30 msgr2=0x7f39200d4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f3928422640 1 --2- 172.21.15.50:0/1661001771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39200c7e30 0x7f39200d4570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39100097a0 tx=0x7f391002b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f3928422640 1 -- 172.21.15.50:0/1661001771 shutdown_connections 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f3928422640 1 --2- 172.21.15.50:0/1661001771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39200c7e30 0x7f39200d4570 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.015+0000 7f3928422640 1 --2- 172.21.15.50:0/1661001771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f392015e3d0 0x7f39200d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.016+0000 7f3928422640 1 -- 172.21.15.50:0/1661001771 >> 172.21.15.50:0/1661001771 conn(0x7f392008dc90 msgr2=0x7f392009d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.016+0000 7f3928422640 1 -- 172.21.15.50:0/1661001771 shutdown_connections 2024-04-02T16:22:45.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.016+0000 7f3928422640 1 -- 172.21.15.50:0/1661001771 wait complete. 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 Processor -- start 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 -- start start 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39200c7e30 0x7f39201f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f392015e3d0 0x7f3920150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392015e7b0 0x7f392015f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f39200d2a00 con 0x7f392015e3d0 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39200d2880 con 0x7f392015e7b0 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f3928422640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f39200d2b80 con 0x7f39200c7e30 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.017+0000 7f39259bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f392015e3d0 0x7f3920150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f392015e3d0 0x7f3920150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44442/0 (socket says 172.21.15.50:44442) 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 -- 172.21.15.50:0/1786202078 learned_addr learned my addr 172.21.15.50:0/1786202078 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39261be640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39200c7e30 0x7f39201f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39269bf640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392015e7b0 0x7f392015f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 -- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39200c7e30 msgr2=0x7f39201f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39200c7e30 0x7f39201f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 -- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392015e7b0 msgr2=0x7f392015f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392015e7b0 0x7f392015f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 -- 172.21.15.50:0/1786202078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39201f45c0 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f39259bd640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f392015e3d0 0x7f3920150840 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f39100098c0 tx=0x7f39100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f3916ffd640 1 -- 172.21.15.50:0/1786202078 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910041070 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f3916ffd640 1 -- 172.21.15.50:0/1786202078 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391003b920 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3920150d80 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f3916ffd640 1 -- 172.21.15.50:0/1786202078 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910035040 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.018+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39200cc560 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.019+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39200d51d0 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.020+0000 7f3916ffd640 1 -- 172.21.15.50:0/1786202078 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f391004c060 con 0x7f392015e3d0 2024-04-02T16:22:45.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.020+0000 7f3916ffd640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f390406c920 0x7f390406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.020+0000 7f3916ffd640 1 -- 172.21.15.50:0/1786202078 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f39100ad820 con 0x7f392015e3d0 2024-04-02T16:22:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.020+0000 7f39261be640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f390406c920 0x7f390406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.021+0000 7f39261be640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f390406c920 0x7f390406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f391c006fd0 tx=0x7f391c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.024+0000 7f3916ffd640 1 -- 172.21.15.50:0/1786202078 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3920150d80 con 0x7f392015e3d0 2024-04-02T16:22:45.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.208+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3920075740 con 0x7f392015e3d0 2024-04-02T16:22:45.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.209+0000 7f3916ffd640 1 -- 172.21.15.50:0/1786202078 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f39201f45c0 con 0x7f392015e3d0 2024-04-02T16:22:45.209 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:45.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f390406c920 msgr2=0x7f390406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f390406c920 0x7f390406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f391c006fd0 tx=0x7f391c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:45.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f392015e3d0 msgr2=0x7f3920150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f392015e3d0 0x7f3920150840 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f39100098c0 tx=0x7f39100046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 shutdown_connections 2024-04-02T16:22:45.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392015e7b0 0x7f392015f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f390406c920 0x7f390406ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f392015e3d0 0x7f3920150840 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 --2- 172.21.15.50:0/1786202078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39200c7e30 0x7f39201f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 >> 172.21.15.50:0/1786202078 conn(0x7f392008dc90 msgr2=0x7f392015c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:45.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 shutdown_connections 2024-04-02T16:22:45.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.211+0000 7f3928422640 1 -- 172.21.15.50:0/1786202078 wait complete. 2024-04-02T16:22:45.222 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:45.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.326+0000 7ffb6ada3640 1 Processor -- start 2024-04-02T16:22:45.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.326+0000 7ffb6ada3640 1 -- start start 2024-04-02T16:22:45.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb6ada3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb640b4700 0x7ffb64070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb6ada3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffb640d0660 0x7ffb640d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb6ada3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb640d1e60 con 0x7ffb6409aa40 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb6ada3640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ffb640b19c0 con 0x7ffb640d0660 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb6ada3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffb640713a0 con 0x7ffb640b4700 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb69340640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffb640d0660 0x7ffb640d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb69340640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffb640d0660 0x7ffb640d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44452/0 (socket says 172.21.15.50:44452) 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb69340640 1 -- 172.21.15.50:0/3469515740 learned_addr learned my addr 172.21.15.50:0/3469515740 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb69340640 1 -- 172.21.15.50:0/3469515740 >> v2:172.21.15.188:3300/0 conn(0x7ffb640b4700 msgr2=0x7ffb64070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb69340640 1 --2- 172.21.15.50:0/3469515740 >> v2:172.21.15.188:3300/0 conn(0x7ffb640b4700 0x7ffb64070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb69340640 1 -- 172.21.15.50:0/3469515740 >> v1:172.21.15.188:6789/0 conn(0x7ffb6409aa40 legacy=0x7ffb6409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:45.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.327+0000 7ffb69340640 1 -- 172.21.15.50:0/3469515740 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64070de0 con 0x7ffb640d0660 2024-04-02T16:22:45.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.328+0000 7ffb69340640 1 --2- 172.21.15.50:0/3469515740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640d0660 0x7ffb640d0af0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffb640749f0 tx=0x7ffb5c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e9452ff6127a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.328+0000 7ffb637fe640 1 -- 172.21.15.50:0/3469515740 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c02ca80 con 0x7ffb640d0660 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.328+0000 7ffb637fe640 1 -- 172.21.15.50:0/3469515740 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c02cc20 con 0x7ffb640d0660 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.328+0000 7ffb637fe640 1 -- 172.21.15.50:0/3469515740 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0343f0 con 0x7ffb640d0660 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 -- 172.21.15.50:0/3469515740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640d0660 msgr2=0x7ffb640d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/3469515740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640d0660 0x7ffb640d0af0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffb640749f0 tx=0x7ffb5c02bee0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 -- 172.21.15.50:0/3469515740 shutdown_connections 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/3469515740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640d0660 0x7ffb640d0af0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/3469515740 >> v2:172.21.15.188:3300/0 conn(0x7ffb640b4700 0x7ffb64070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 -- 172.21.15.50:0/3469515740 >> 172.21.15.50:0/3469515740 conn(0x7ffb640b3be0 msgr2=0x7ffb640b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 -- 172.21.15.50:0/3469515740 shutdown_connections 2024-04-02T16:22:45.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.329+0000 7ffb6ada3640 1 -- 172.21.15.50:0/3469515740 wait complete. 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.330+0000 7ffb6ada3640 1 Processor -- start 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.330+0000 7ffb6ada3640 1 -- start start 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb6ada3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb6409aa40 0x7ffb641e0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb6ada3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640b4700 0x7ffb641df800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb6ada3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb640d0660 0x7ffb641e75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb6ada3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6406c9b0 con 0x7ffb640b4700 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb6ada3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6406c6d0 con 0x7ffb640d0660 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb6ada3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6406cc90 con 0x7ffb6409aa40 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640b4700 0x7ffb641df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640b4700 0x7ffb641df800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44456/0 (socket says 172.21.15.50:44456) 2024-04-02T16:22:45.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 -- 172.21.15.50:0/1108477955 learned_addr learned my addr 172.21.15.50:0/1108477955 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb69340640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb640d0660 0x7ffb641e75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb68b3f640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb6409aa40 0x7ffb641e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 -- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb6409aa40 msgr2=0x7ffb641e0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb6409aa40 0x7ffb641e0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 -- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb640d0660 msgr2=0x7ffb641e75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb640d0660 0x7ffb641e75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.331+0000 7ffb63fff640 1 -- 172.21.15.50:0/1108477955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641e60c0 con 0x7ffb640b4700 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.332+0000 7ffb63fff640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640b4700 0x7ffb641df800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b480 tx=0x7ffb5800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.332+0000 7ffb617fa640 1 -- 172.21.15.50:0/1108477955 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58014070 con 0x7ffb640b4700 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.332+0000 7ffb617fa640 1 -- 172.21.15.50:0/1108477955 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb58010870 con 0x7ffb640b4700 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.332+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb641e63a0 con 0x7ffb640b4700 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.332+0000 7ffb617fa640 1 -- 172.21.15.50:0/1108477955 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58010c60 con 0x7ffb640b4700 2024-04-02T16:22:45.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.332+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb641e6db0 con 0x7ffb640b4700 2024-04-02T16:22:45.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.333+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb640d1c80 con 0x7ffb640b4700 2024-04-02T16:22:45.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.333+0000 7ffb617fa640 1 -- 172.21.15.50:0/1108477955 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb58017070 con 0x7ffb640b4700 2024-04-02T16:22:45.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.334+0000 7ffb617fa640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb5006c8f0 0x7ffb5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.334+0000 7ffb617fa640 1 -- 172.21.15.50:0/1108477955 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ffb5808bcc0 con 0x7ffb640b4700 2024-04-02T16:22:45.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.334+0000 7ffb68b3f640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb5006c8f0 0x7ffb5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.334+0000 7ffb68b3f640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb5006c8f0 0x7ffb5006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb640b2d80 tx=0x7ffb54009060 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.338+0000 7ffb617fa640 1 -- 172.21.15.50:0/1108477955 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffb640d1c80 con 0x7ffb640b4700 2024-04-02T16:22:45.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.526+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb64075760 con 0x7ffb640b4700 2024-04-02T16:22:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.526+0000 7ffb617fa640 1 -- 172.21.15.50:0/1108477955 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ffb58059710 con 0x7ffb640b4700 2024-04-02T16:22:45.527 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.528+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb5006c8f0 msgr2=0x7ffb5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb5006c8f0 0x7ffb5006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb640b2d80 tx=0x7ffb54009060 comp rx=0 tx=0).stop 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640b4700 msgr2=0x7ffb641df800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640b4700 0x7ffb641df800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b480 tx=0x7ffb5800b940 comp rx=0 tx=0).stop 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 shutdown_connections 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb640d0660 0x7ffb641e75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb5006c8f0 0x7ffb5006edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb640b4700 0x7ffb641df800 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 --2- 172.21.15.50:0/1108477955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb6409aa40 0x7ffb641e0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 >> 172.21.15.50:0/1108477955 conn(0x7ffb640b3be0 msgr2=0x7ffb641e5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 shutdown_connections 2024-04-02T16:22:45.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.529+0000 7ffb6ada3640 1 -- 172.21.15.50:0/1108477955 wait complete. 2024-04-02T16:22:45.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:45.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.643+0000 7f618f6eb640 1 Processor -- start 2024-04-02T16:22:45.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.644+0000 7f618f6eb640 1 -- start start 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.644+0000 7f618f6eb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6188140fc0 0x7f61880cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.644+0000 7f618f6eb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61880cef40 0x7f6188071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.644+0000 7f618f6eb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6188141690 con 0x7f618813e0b0 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.644+0000 7f618f6eb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6188071360 con 0x7f61880cef40 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.644+0000 7f618f6eb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f61880cf3f0 con 0x7f6188140fc0 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618dc88640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61880cef40 0x7f6188071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618cc86640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6188140fc0 0x7f61880cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618d487640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f618813e0b0 0x7f61881408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:38758/0 (socket says 172.21.15.50:38758) 2024-04-02T16:22:45.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618d487640 1 -- 172.21.15.50:0/2844374950 learned_addr learned my addr 172.21.15.50:0/2844374950 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:45.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618cc86640 1 -- 172.21.15.50:0/2844374950 >> v2:172.21.15.188:3300/0 conn(0x7f61880cef40 msgr2=0x7f6188071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:45.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618cc86640 1 --2- 172.21.15.50:0/2844374950 >> v2:172.21.15.188:3300/0 conn(0x7f61880cef40 0x7f6188071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618cc86640 1 -- 172.21.15.50:0/2844374950 >> v1:172.21.15.188:6789/0 conn(0x7f618813e0b0 legacy=0x7f61881408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618cc86640 1 -- 172.21.15.50:0/2844374950 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61880cf670 con 0x7f6188140fc0 2024-04-02T16:22:45.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.645+0000 7f618dc88640 1 --2- 172.21.15.50:0/2844374950 >> v2:172.21.15.188:3300/0 conn(0x7f61880cef40 0x7f6188071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.646+0000 7f618cc86640 1 --2- 172.21.15.50:0/2844374950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6188140fc0 0x7f61880cea00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61780097a0 tx=0x7f617802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84b89366d126b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.646+0000 7f617ffff640 1 -- 172.21.15.50:0/2844374950 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178035020 con 0x7f6188140fc0 2024-04-02T16:22:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.646+0000 7f617ffff640 1 -- 172.21.15.50:0/2844374950 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6178031830 con 0x7f6188140fc0 2024-04-02T16:22:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.646+0000 7f617ffff640 1 -- 172.21.15.50:0/2844374950 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178031c00 con 0x7f6188140fc0 2024-04-02T16:22:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.646+0000 7f618f6eb640 1 -- 172.21.15.50:0/2844374950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6188140fc0 msgr2=0x7f61880cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.646+0000 7f618f6eb640 1 --2- 172.21.15.50:0/2844374950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6188140fc0 0x7f61880cea00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61780097a0 tx=0x7f617802b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.647+0000 7f618f6eb640 1 -- 172.21.15.50:0/2844374950 shutdown_connections 2024-04-02T16:22:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.647+0000 7f618f6eb640 1 --2- 172.21.15.50:0/2844374950 >> v2:172.21.15.188:3300/0 conn(0x7f61880cef40 0x7f6188071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.647+0000 7f618f6eb640 1 --2- 172.21.15.50:0/2844374950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6188140fc0 0x7f61880cea00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.647+0000 7f618f6eb640 1 -- 172.21.15.50:0/2844374950 >> 172.21.15.50:0/2844374950 conn(0x7f618806eb90 msgr2=0x7f618806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.647+0000 7f618f6eb640 1 -- 172.21.15.50:0/2844374950 shutdown_connections 2024-04-02T16:22:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.647+0000 7f618f6eb640 1 -- 172.21.15.50:0/2844374950 wait complete. 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.648+0000 7f618f6eb640 1 Processor -- start 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.648+0000 7f618f6eb640 1 -- start start 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.648+0000 7f618f6eb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61880cef40 0x7f61880722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.648+0000 7f618f6eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618813e0b0 0x7f6188143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618f6eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6188140fc0 0x7f6188137420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618f6eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61880715b0 con 0x7f6188140fc0 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618f6eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61880cfc30 con 0x7f618813e0b0 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618f6eb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f618813e490 con 0x7f61880cef40 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6188140fc0 0x7f6188137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6188140fc0 0x7f6188137420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37896/0 (socket says 172.21.15.50:37896) 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 -- 172.21.15.50:0/4208546158 learned_addr learned my addr 172.21.15.50:0/4208546158 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:45.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618d487640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61880cef40 0x7f61880722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618cc86640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618813e0b0 0x7f6188143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 -- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61880cef40 msgr2=0x7f61880722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61880cef40 0x7f61880722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 -- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618813e0b0 msgr2=0x7f6188143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618813e0b0 0x7f6188143d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 -- 172.21.15.50:0/4208546158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6188144790 con 0x7f6188140fc0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618dc88640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6188140fc0 0x7f6188137420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6180009f00 tx=0x7f618000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f617dffb640 1 -- 172.21.15.50:0/4208546158 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180018030 con 0x7f6188140fc0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f617dffb640 1 -- 172.21.15.50:0/4208546158 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618000ce50 con 0x7f6188140fc0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6188142b90 con 0x7f6188140fc0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.649+0000 7f617dffb640 1 -- 172.21.15.50:0/4208546158 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180003bb0 con 0x7f6188140fc0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.650+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6188143150 con 0x7f6188140fc0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.651+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6188075760 con 0x7f6188140fc0 2024-04-02T16:22:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.651+0000 7f617dffb640 1 -- 172.21.15.50:0/4208546158 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6180005d10 con 0x7f6188140fc0 2024-04-02T16:22:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.651+0000 7f617dffb640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f616c06c8f0 0x7f616c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.651+0000 7f617dffb640 1 -- 172.21.15.50:0/4208546158 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f618008c900 con 0x7f6188140fc0 2024-04-02T16:22:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.652+0000 7f618d487640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f616c06c8f0 0x7f616c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.652+0000 7f618d487640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f616c06c8f0 0x7f616c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6188144b90 tx=0x7f6184007e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.656+0000 7f617dffb640 1 -- 172.21.15.50:0/4208546158 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6180022070 con 0x7f6188140fc0 2024-04-02T16:22:45.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.842+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f61880734d0 con 0x7f6188140fc0 2024-04-02T16:22:45.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.843+0000 7f617dffb640 1 -- 172.21.15.50:0/4208546158 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f6180010040 con 0x7f6188140fc0 2024-04-02T16:22:45.843 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:45.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f616c06c8f0 msgr2=0x7f616c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f616c06c8f0 0x7f616c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6188144b90 tx=0x7f6184007e80 comp rx=0 tx=0).stop 2024-04-02T16:22:45.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6188140fc0 msgr2=0x7f6188137420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6188140fc0 0x7f6188137420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6180009f00 tx=0x7f618000c490 comp rx=0 tx=0).stop 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 shutdown_connections 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f616c06c8f0 0x7f616c06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6188140fc0 0x7f6188137420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f618813e0b0 0x7f6188143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 --2- 172.21.15.50:0/4208546158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61880cef40 0x7f61880722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 >> 172.21.15.50:0/4208546158 conn(0x7f618806eb90 msgr2=0x7f618815cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 shutdown_connections 2024-04-02T16:22:45.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.845+0000 7f618f6eb640 1 -- 172.21.15.50:0/4208546158 wait complete. 2024-04-02T16:22:45.856 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:45.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.958+0000 7ff7064fb640 1 Processor -- start 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.958+0000 7ff7064fb640 1 -- start start 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff7064fb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff70009aa40 0x7ff70009ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff7064fb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7000b45b0 0x7ff7000cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff7064fb640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff70009b360 con 0x7ff7000cd610 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff7064fb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ff70006f840 con 0x7ff70009aa40 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff7064fb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff70006fb20 con 0x7ff7000b45b0 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff6fffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff70009aa40 0x7ff70009ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff6fffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff70009aa40 0x7ff70009ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37910/0 (socket says 172.21.15.50:37910) 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff6fffff640 1 -- 172.21.15.50:0/2702491830 learned_addr learned my addr 172.21.15.50:0/2702491830 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff704a98640 1 --1- 172.21.15.50:0/2702491830 >> v1:172.21.15.50:6789/0 conn(0x7ff7000cd610 0x7ff7000cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34598/0 (socket says 172.21.15.50:34598) 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.959+0000 7ff6ff7fe640 1 --2- 172.21.15.50:0/2702491830 >> v2:172.21.15.188:3300/0 conn(0x7ff7000b45b0 0x7ff7000cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6fffff640 1 -- 172.21.15.50:0/2702491830 >> v2:172.21.15.188:3300/0 conn(0x7ff7000b45b0 msgr2=0x7ff7000cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6fffff640 1 --2- 172.21.15.50:0/2702491830 >> v2:172.21.15.188:3300/0 conn(0x7ff7000b45b0 0x7ff7000cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6fffff640 1 -- 172.21.15.50:0/2702491830 >> v1:172.21.15.50:6789/0 conn(0x7ff7000cd610 legacy=0x7ff7000cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6fffff640 1 -- 172.21.15.50:0/2702491830 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7000cc5f0 con 0x7ff70009aa40 2024-04-02T16:22:45.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6fffff640 1 --2- 172.21.15.50:0/2702491830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff70009aa40 0x7ff70009ae20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff70006f350 tx=0x7ff6e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b9fea08d127f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6feffd640 1 -- 172.21.15.50:0/2702491830 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8034070 con 0x7ff70009aa40 2024-04-02T16:22:45.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6feffd640 1 -- 172.21.15.50:0/2702491830 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e8030860 con 0x7ff70009aa40 2024-04-02T16:22:45.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.960+0000 7ff6feffd640 1 -- 172.21.15.50:0/2702491830 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8030c30 con 0x7ff70009aa40 2024-04-02T16:22:45.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 -- 172.21.15.50:0/2702491830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff70009aa40 msgr2=0x7ff70009ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 --2- 172.21.15.50:0/2702491830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff70009aa40 0x7ff70009ae20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff70006f350 tx=0x7ff6e802b9a0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 -- 172.21.15.50:0/2702491830 shutdown_connections 2024-04-02T16:22:45.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 --2- 172.21.15.50:0/2702491830 >> v2:172.21.15.188:3300/0 conn(0x7ff7000b45b0 0x7ff7000cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 --2- 172.21.15.50:0/2702491830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff70009aa40 0x7ff70009ae20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 -- 172.21.15.50:0/2702491830 >> 172.21.15.50:0/2702491830 conn(0x7ff7000b3a90 msgr2=0x7ff7000b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 -- 172.21.15.50:0/2702491830 shutdown_connections 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.961+0000 7ff7064fb640 1 -- 172.21.15.50:0/2702491830 wait complete. 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.962+0000 7ff7064fb640 1 Processor -- start 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.962+0000 7ff7064fb640 1 -- start start 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff7064fb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff70009aa40 0x7ff7000b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff7064fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7000b45b0 0x7ff7000c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff7064fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7000cd610 0x7ff70014f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff7064fb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff70009b860 con 0x7ff7000cd610 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff7064fb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7000ccbb0 con 0x7ff7000b45b0 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff7064fb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff70009b9e0 con 0x7ff70009aa40 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7000cd610 0x7ff70014f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7000cd610 0x7ff70014f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37914/0 (socket says 172.21.15.50:37914) 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 -- 172.21.15.50:0/3489524393 learned_addr learned my addr 172.21.15.50:0/3489524393 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:45.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff6ff7fe640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7000b45b0 0x7ff7000c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff6fffff640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff70009aa40 0x7ff7000b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 -- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff70009aa40 msgr2=0x7ff7000b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff70009aa40 0x7ff7000b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 -- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7000b45b0 msgr2=0x7ff7000c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7000b45b0 0x7ff7000c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff704a98640 1 -- 172.21.15.50:0/3489524393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7000c9b10 con 0x7ff7000cd610 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.963+0000 7ff6ff7fe640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7000b45b0 0x7ff7000c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.964+0000 7ff704a98640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7000cd610 0x7ff70014f000 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c710 tx=0x7ff6f800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.964+0000 7ff6fcff9640 1 -- 172.21.15.50:0/3489524393 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8017070 con 0x7ff7000cd610 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.964+0000 7ff6fcff9640 1 -- 172.21.15.50:0/3489524393 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f800ce20 con 0x7ff7000cd610 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.964+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7000c9df0 con 0x7ff7000cd610 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.964+0000 7ff6fcff9640 1 -- 172.21.15.50:0/3489524393 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800a860 con 0x7ff7000cd610 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.964+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7000ca3b0 con 0x7ff7000cd610 2024-04-02T16:22:45.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.965+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7000756f0 con 0x7ff7000cd610 2024-04-02T16:22:45.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.965+0000 7ff6fcff9640 1 -- 172.21.15.50:0/3489524393 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6f801a070 con 0x7ff7000cd610 2024-04-02T16:22:45.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.966+0000 7ff6fcff9640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:45.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.966+0000 7ff6fcff9640 1 -- 172.21.15.50:0/3489524393 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff6f808b600 con 0x7ff7000cd610 2024-04-02T16:22:45.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.966+0000 7ff6fffff640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:45.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.967+0000 7ff6fffff640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff6e006c8f0 0x7ff6e006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036000 tx=0x7ff6e803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:45.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:45.970+0000 7ff6fcff9640 1 -- 172.21.15.50:0/3489524393 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff6f8059610 con 0x7ff7000cd610 2024-04-02T16:22:46.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.155+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff700074ec0 con 0x7ff7000cd610 2024-04-02T16:22:46.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.157+0000 7ff6fcff9640 1 -- 172.21.15.50:0/3489524393 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7ff6f8058f10 con 0x7ff7000cd610 2024-04-02T16:22:46.157 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:46.157 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:46.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff6e006c8f0 msgr2=0x7ff6e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff6e006c8f0 0x7ff6e006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036000 tx=0x7ff6e803b040 comp rx=0 tx=0).stop 2024-04-02T16:22:46.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7000cd610 msgr2=0x7ff70014f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7000cd610 0x7ff70014f000 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c710 tx=0x7ff6f800cbd0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 shutdown_connections 2024-04-02T16:22:46.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7000cd610 0x7ff70014f000 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7000b45b0 0x7ff7000c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 --2- 172.21.15.50:0/3489524393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff70009aa40 0x7ff7000b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 >> 172.21.15.50:0/3489524393 conn(0x7ff7000b3a90 msgr2=0x7ff70015c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 shutdown_connections 2024-04-02T16:22:46.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.159+0000 7ff7064fb640 1 -- 172.21.15.50:0/3489524393 wait complete. 2024-04-02T16:22:46.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:46.171 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:46.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.275+0000 7f835ec88640 1 Processor -- start 2024-04-02T16:22:46.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.275+0000 7f835ec88640 1 -- start start 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f835ec88640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8358144bd0 0x7f8358072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f835ec88640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f835ec88640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8358073350 con 0x7f83580b58b0 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f835ec88640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f83580ce760 con 0x7f8358144bd0 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f835ec88640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8358072880 con 0x7f83580b33c0 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f835ca24640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8358144bd0 0x7f8358072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f835d225640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f8357fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f83580b58b0 0x7f83580b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:42736/0 (socket says 172.21.15.50:42736) 2024-04-02T16:22:46.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.276+0000 7f8357fff640 1 -- 172.21.15.50:0/3968069056 learned_addr learned my addr 172.21.15.50:0/3968069056 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:46.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.277+0000 7f835d225640 1 -- 172.21.15.50:0/3968069056 >> v2:172.21.15.188:3300/0 conn(0x7f8358144bd0 msgr2=0x7f8358072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:46.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.277+0000 7f835d225640 1 --2- 172.21.15.50:0/3968069056 >> v2:172.21.15.188:3300/0 conn(0x7f8358144bd0 0x7f8358072340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.277+0000 7f835d225640 1 -- 172.21.15.50:0/3968069056 >> v1:172.21.15.188:6789/0 conn(0x7f83580b58b0 legacy=0x7f83580b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.277+0000 7f835d225640 1 -- 172.21.15.50:0/3968069056 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8358072ab0 con 0x7f83580b33c0 2024-04-02T16:22:46.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.277+0000 7f835ca24640 1 --2- 172.21.15.50:0/3968069056 >> v2:172.21.15.188:3300/0 conn(0x7f8358144bd0 0x7f8358072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835d225640 1 --2- 172.21.15.50:0/3968069056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580ceeb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8350009d90 tx=0x7f835002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8841deb798049eda server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f83577fe640 1 -- 172.21.15.50:0/3968069056 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350039070 con 0x7f83580b33c0 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f83577fe640 1 -- 172.21.15.50:0/3968069056 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835002cc50 con 0x7f83580b33c0 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f83577fe640 1 -- 172.21.15.50:0/3968069056 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350034440 con 0x7f83580b33c0 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835ec88640 1 -- 172.21.15.50:0/3968069056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 msgr2=0x7f83580ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835ec88640 1 --2- 172.21.15.50:0/3968069056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580ceeb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8350009d90 tx=0x7f835002bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835ec88640 1 -- 172.21.15.50:0/3968069056 shutdown_connections 2024-04-02T16:22:46.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835ec88640 1 --2- 172.21.15.50:0/3968069056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580ceeb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835ec88640 1 --2- 172.21.15.50:0/3968069056 >> v2:172.21.15.188:3300/0 conn(0x7f8358144bd0 0x7f8358072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835ec88640 1 -- 172.21.15.50:0/3968069056 >> 172.21.15.50:0/3968069056 conn(0x7f835806e6c0 msgr2=0x7f835806eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:46.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.278+0000 7f835ec88640 1 -- 172.21.15.50:0/3968069056 shutdown_connections 2024-04-02T16:22:46.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.279+0000 7f835ec88640 1 -- 172.21.15.50:0/3968069056 wait complete. 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.280+0000 7f835ec88640 1 Processor -- start 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.280+0000 7f835ec88640 1 -- start start 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.280+0000 7f835ec88640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.280+0000 7f835ec88640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580b58b0 0x7f835813c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835ec88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8358144bd0 0x7f835806b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835ec88640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f83580ce990 con 0x7f8358144bd0 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835ec88640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8358073070 con 0x7f83580b58b0 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835ec88640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f83580b5d10 con 0x7f83580b33c0 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8358144bd0 0x7f835806b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8358144bd0 0x7f835806b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37930/0 (socket says 172.21.15.50:37930) 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 -- 172.21.15.50:0/2757810057 learned_addr learned my addr 172.21.15.50:0/2757810057 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835ca24640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f8357fff640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580b58b0 0x7f835813c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 -- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 msgr2=0x7f83580b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 -- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580b58b0 msgr2=0x7f835813c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580b58b0 0x7f835813c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 -- 172.21.15.50:0/2757810057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83580ab850 con 0x7f8358144bd0 2024-04-02T16:22:46.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835d225640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8358144bd0 0x7f835806b520 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8350009d90 tx=0x7f8350004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f83557fa640 1 -- 172.21.15.50:0/2757810057 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350039050 con 0x7f8358144bd0 2024-04-02T16:22:46.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f83557fa640 1 -- 172.21.15.50:0/2757810057 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8350045070 con 0x7f8358144bd0 2024-04-02T16:22:46.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83580ae3d0 con 0x7f8358144bd0 2024-04-02T16:22:46.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f83557fa640 1 -- 172.21.15.50:0/2757810057 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835003e610 con 0x7f8358144bd0 2024-04-02T16:22:46.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.281+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83580abaf0 con 0x7f8358144bd0 2024-04-02T16:22:46.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.282+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358074a10 con 0x7f8358144bd0 2024-04-02T16:22:46.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.283+0000 7f83557fa640 1 -- 172.21.15.50:0/2757810057 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f835003e7b0 con 0x7f8358144bd0 2024-04-02T16:22:46.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.283+0000 7f83557fa640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f833c06c8f0 0x7f833c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.283+0000 7f83557fa640 1 -- 172.21.15.50:0/2757810057 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f83500ae900 con 0x7f8358144bd0 2024-04-02T16:22:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.283+0000 7f835ca24640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f833c06c8f0 0x7f833c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.284+0000 7f835ca24640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f833c06c8f0 0x7f833c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f834c009560 tx=0x7f834c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.287+0000 7f83557fa640 1 -- 172.21.15.50:0/2757810057 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f835007c350 con 0x7f8358144bd0 2024-04-02T16:22:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.470+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8358075760 con 0x7f8358144bd0 2024-04-02T16:22:46.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.471+0000 7f83557fa640 1 -- 172.21.15.50:0/2757810057 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f8350080650 con 0x7f8358144bd0 2024-04-02T16:22:46.471 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f833c06c8f0 msgr2=0x7f833c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f833c06c8f0 0x7f833c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f834c009560 tx=0x7f834c009190 comp rx=0 tx=0).stop 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8358144bd0 msgr2=0x7f835806b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8358144bd0 0x7f835806b520 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8350009d90 tx=0x7f8350004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 shutdown_connections 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f833c06c8f0 0x7f833c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8358144bd0 0x7f835806b520 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580b58b0 0x7f835813c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 --2- 172.21.15.50:0/2757810057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83580b33c0 0x7f83580b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 >> 172.21.15.50:0/2757810057 conn(0x7f835806e6c0 msgr2=0x7f83580cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 shutdown_connections 2024-04-02T16:22:46.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.473+0000 7f835ec88640 1 -- 172.21.15.50:0/2757810057 wait complete. 2024-04-02T16:22:46.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:46.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 Processor -- start 2024-04-02T16:22:46.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 -- start start 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe13c144bd0 0x7fe13c072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe13c0b58b0 0x7fe13c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c0b31f0 0x7fe13c06ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fe13c0ce760 con 0x7fe13c0b58b0 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe13c072880 con 0x7fe13c144bd0 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.589+0000 7fe142b40640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe13c073350 con 0x7fe13c0b31f0 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe13c0b58b0 0x7fe13c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe13c0b58b0 0x7fe13c0b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37936/0 (socket says 172.21.15.50:37936) 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 -- 172.21.15.50:0/1362049931 learned_addr learned my addr 172.21.15.50:0/1362049931 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe1408dc640 1 --2- 172.21.15.50:0/1362049931 >> v2:172.21.15.188:3300/0 conn(0x7fe13c144bd0 0x7fe13c072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe1410dd640 1 --2- 172.21.15.50:0/1362049931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c0b31f0 0x7fe13c06ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 -- 172.21.15.50:0/1362049931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c0b31f0 msgr2=0x7fe13c06ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 --2- 172.21.15.50:0/1362049931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c0b31f0 0x7fe13c06ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 -- 172.21.15.50:0/1362049931 >> v2:172.21.15.188:3300/0 conn(0x7fe13c144bd0 msgr2=0x7fe13c072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:46.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 --2- 172.21.15.50:0/1362049931 >> v2:172.21.15.188:3300/0 conn(0x7fe13c144bd0 0x7fe13c072340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.590+0000 7fe13bfff640 1 -- 172.21.15.50:0/1362049931 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c072ab0 con 0x7fe13c0b58b0 2024-04-02T16:22:46.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe13bfff640 1 --2- 172.21.15.50:0/1362049931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b58b0 0x7fe13c0b2cb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe13c075010 tx=0x7fe12802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9938cef25c91ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe13b7fe640 1 -- 172.21.15.50:0/1362049931 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128035020 con 0x7fe13c0b58b0 2024-04-02T16:22:46.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe13b7fe640 1 -- 172.21.15.50:0/1362049931 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe128031870 con 0x7fe13c0b58b0 2024-04-02T16:22:46.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe13b7fe640 1 -- 172.21.15.50:0/1362049931 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128031c40 con 0x7fe13c0b58b0 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 -- 172.21.15.50:0/1362049931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b58b0 msgr2=0x7fe13c0b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 --2- 172.21.15.50:0/1362049931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b58b0 0x7fe13c0b2cb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe13c075010 tx=0x7fe12802b910 comp rx=0 tx=0).stop 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 -- 172.21.15.50:0/1362049931 shutdown_connections 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 --2- 172.21.15.50:0/1362049931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c0b31f0 0x7fe13c06ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 --2- 172.21.15.50:0/1362049931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b58b0 0x7fe13c0b2cb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 --2- 172.21.15.50:0/1362049931 >> v2:172.21.15.188:3300/0 conn(0x7fe13c144bd0 0x7fe13c072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 -- 172.21.15.50:0/1362049931 >> 172.21.15.50:0/1362049931 conn(0x7fe13c06e6c0 msgr2=0x7fe13c06eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 -- 172.21.15.50:0/1362049931 shutdown_connections 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.591+0000 7fe142b40640 1 -- 172.21.15.50:0/1362049931 wait complete. 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.592+0000 7fe142b40640 1 Processor -- start 2024-04-02T16:22:46.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.592+0000 7fe142b40640 1 -- start start 2024-04-02T16:22:46.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe142b40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b31f0 0x7fe13c134960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe142b40640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe13c0b58b0 0x7fe13c1e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe142b40640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c144bd0 0x7fe13c132e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe142b40640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c0ce990 con 0x7fe13c0b31f0 2024-04-02T16:22:46.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe142b40640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c073070 con 0x7fe13c0b58b0 2024-04-02T16:22:46.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe142b40640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe13c0b5d10 con 0x7fe13c144bd0 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe1408dc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b31f0 0x7fe13c134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe1408dc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b31f0 0x7fe13c134960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37946/0 (socket says 172.21.15.50:37946) 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.593+0000 7fe1408dc640 1 -- 172.21.15.50:0/913423248 learned_addr learned my addr 172.21.15.50:0/913423248 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1410dd640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c144bd0 0x7fe13c132e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe13bfff640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe13c0b58b0 0x7fe13c1e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1408dc640 1 -- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c144bd0 msgr2=0x7fe13c132e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1408dc640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c144bd0 0x7fe13c132e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1408dc640 1 -- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe13c0b58b0 msgr2=0x7fe13c1e1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1408dc640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe13c0b58b0 0x7fe13c1e1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1408dc640 1 -- 172.21.15.50:0/913423248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c1338c0 con 0x7fe13c0b31f0 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1410dd640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c144bd0 0x7fe13c132e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe13bfff640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe13c0b58b0 0x7fe13c1e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1408dc640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b31f0 0x7fe13c134960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe13000d620 tx=0x7fe13000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1397fa640 1 -- 172.21.15.50:0/913423248 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13000b620 con 0x7fe13c0b31f0 2024-04-02T16:22:46.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1397fa640 1 -- 172.21.15.50:0/913423248 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe130019070 con 0x7fe13c0b31f0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c133ba0 con 0x7fe13c0b31f0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe1397fa640 1 -- 172.21.15.50:0/913423248 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13000ba00 con 0x7fe13c0b31f0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.594+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c134440 con 0x7fe13c0b31f0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.595+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13c133d70 con 0x7fe13c0b31f0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.596+0000 7fe1397fa640 1 -- 172.21.15.50:0/913423248 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe13000bba0 con 0x7fe13c0b31f0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.596+0000 7fe1397fa640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe12006c8f0 0x7fe12006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.596+0000 7fe1397fa640 1 -- 172.21.15.50:0/913423248 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fe13008c490 con 0x7fe13c0b31f0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.596+0000 7fe13bfff640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe12006c8f0 0x7fe12006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.597+0000 7fe13bfff640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe12006c8f0 0x7fe12006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe128036000 tx=0x7fe128008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.600+0000 7fe1397fa640 1 -- 172.21.15.50:0/913423248 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe13005a400 con 0x7fe13c0b31f0 2024-04-02T16:22:46.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.786+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe13c074a10 con 0x7fe13c0b31f0 2024-04-02T16:22:46.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.786+0000 7fe1397fa640 1 -- 172.21.15.50:0/913423248 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fe130059d00 con 0x7fe13c0b31f0 2024-04-02T16:22:46.787 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe12006c8f0 msgr2=0x7fe12006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe12006c8f0 0x7fe12006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe128036000 tx=0x7fe128008030 comp rx=0 tx=0).stop 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b31f0 msgr2=0x7fe13c134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b31f0 0x7fe13c134960 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe13000d620 tx=0x7fe13000dae0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 shutdown_connections 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe13c144bd0 0x7fe13c132e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe13c0b58b0 0x7fe13c1e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe12006c8f0 0x7fe12006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 --2- 172.21.15.50:0/913423248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe13c0b31f0 0x7fe13c134960 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 >> 172.21.15.50:0/913423248 conn(0x7fe13c06e6c0 msgr2=0x7fe13c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:46.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 shutdown_connections 2024-04-02T16:22:46.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.789+0000 7fe142b40640 1 -- 172.21.15.50:0/913423248 wait complete. 2024-04-02T16:22:46.800 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:46.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.904+0000 7fa8f0455640 1 Processor -- start 2024-04-02T16:22:46.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.904+0000 7fa8f0455640 1 -- start start 2024-04-02T16:22:46.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8f0455640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8e815e3d0 0x7fa8e80d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8f0455640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e80d2f30 con 0x7fa8e815e7b0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8f0455640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e80cf000 con 0x7fa8e80c7e30 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8f0455640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e80cf2e0 con 0x7fa8e815e3d0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ee9f2640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fa8e815e7b0 0x7fa8e80cfae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34614/0 (socket says 172.21.15.50:34614) 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ee9f2640 1 -- 172.21.15.50:0/3825861150 learned_addr learned my addr 172.21.15.50:0/3825861150 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3668739404 0 0) 0x7fa8e80d2f30 con 0x7fa8e815e7b0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8d4003520 con 0x7fa8e815e7b0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ee1f1640 1 --2- 172.21.15.50:0/3825861150 >> v2:172.21.15.188:3300/0 conn(0x7fa8e815e3d0 0x7fa8e80d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2092608799 0 0) 0x7fa8d4003520 con 0x7fa8e815e7b0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 >> v2:172.21.15.188:3300/0 conn(0x7fa8e815e3d0 msgr2=0x7fa8e80d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 --2- 172.21.15.50:0/3825861150 >> v2:172.21.15.188:3300/0 conn(0x7fa8e815e3d0 0x7fa8e80d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 >> v1:172.21.15.188:6789/0 conn(0x7fa8e80c7e30 legacy=0x7fa8e80d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e80cf5c0 con 0x7fa8e815e7b0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fa8e00019f0 con 0x7fa8e815e7b0 2024-04-02T16:22:46.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 >> v1:172.21.15.50:6789/0 conn(0x7fa8e815e7b0 legacy=0x7fa8e80cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d40039b0 0x7fa8d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8d4004800 0x7fa8d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8d40051c0 0x7fa8d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e80cf2e0 con 0x7fa8d4004800 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d400c060 con 0x7fa8d40039b0 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.905+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d400c260 con 0x7fa8d40051c0 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ed9f0640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8d4004800 0x7fa8d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ee1f1640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d40039b0 0x7fa8d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ed9f0640 1 -- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8d40051c0 msgr2=0x7fa8d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ee9f2640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8d40051c0 0x7fa8d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ed9f0640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8d40051c0 0x7fa8d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ed9f0640 1 -- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d40039b0 msgr2=0x7fa8d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ed9f0640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d40039b0 0x7fa8d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ed9f0640 1 -- 172.21.15.50:0/3825861150 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d4003520 con 0x7fa8d4004800 2024-04-02T16:22:46.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ee1f1640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d40039b0 0x7fa8d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ee9f2640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8d40051c0 0x7fa8d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.906+0000 7fa8ee9f2640 1 -- 172.21.15.50:0/3825861150 reap_dead start 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8ed9f0640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8d4004800 0x7fa8d4004c80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8001720 tx=0x7fa8d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ac322af97544de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8001370 con 0x7fa8d4004800 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d80052d0 con 0x7fa8d4004800 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8ed1ef640 1 -- 172.21.15.50:0/3825861150 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8003830 con 0x7fa8d4004800 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8f0455640 1 -- 172.21.15.50:0/3825861150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8d4004800 msgr2=0x7fa8d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8d4004800 0x7fa8d4004c80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8001720 tx=0x7fa8d4005830 comp rx=0 tx=0).stop 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8f0455640 1 -- 172.21.15.50:0/3825861150 shutdown_connections 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3825861150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8d4004800 0x7fa8d4004c80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8f0455640 1 -- 172.21.15.50:0/3825861150 >> 172.21.15.50:0/3825861150 conn(0x7fa8e808dc90 msgr2=0x7fa8e809d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8f0455640 1 -- 172.21.15.50:0/3825861150 shutdown_connections 2024-04-02T16:22:46.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.907+0000 7fa8f0455640 1 -- 172.21.15.50:0/3825861150 wait complete. 2024-04-02T16:22:46.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.908+0000 7fa8f0455640 1 Processor -- start 2024-04-02T16:22:46.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8f0455640 1 -- start start 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8f0455640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d4004800 0x7fa8e814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8f0455640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e80b8da0 0x7fa8e80b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8f0455640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8e815ed50 0x7fa8e815f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8f0455640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e80d41b0 con 0x7fa8e80b8da0 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8f0455640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e80d4030 con 0x7fa8d4004800 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8f0455640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e80d4330 con 0x7fa8e815ed50 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8ed9f0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e80b8da0 0x7fa8e80b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8ed9f0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e80b8da0 0x7fa8e80b9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37964/0 (socket says 172.21.15.50:37964) 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.909+0000 7fa8ed9f0640 1 -- 172.21.15.50:0/3928875447 learned_addr learned my addr 172.21.15.50:0/3928875447 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ee1f1640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d4004800 0x7fa8e814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ee9f2640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8e815ed50 0x7fa8e815f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ed9f0640 1 -- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8e815ed50 msgr2=0x7fa8e815f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ed9f0640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8e815ed50 0x7fa8e815f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ed9f0640 1 -- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d4004800 msgr2=0x7fa8e814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ed9f0640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d4004800 0x7fa8e814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ed9f0640 1 -- 172.21.15.50:0/3928875447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e815fb10 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8ed9f0640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e80b8da0 0x7fa8e80b9220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8000c00 tx=0x7fa8d8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8deffd640 1 -- 172.21.15.50:0/3928875447 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8041020 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8deffd640 1 -- 172.21.15.50:0/3928875447 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d8005480 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e815fdf0 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8deffd640 1 -- 172.21.15.50:0/3928875447 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8007a30 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.910+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e81603b0 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.911+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e8075760 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.912+0000 7fa8deffd640 1 -- 172.21.15.50:0/3928875447 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8d8044020 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.912+0000 7fa8deffd640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa8cc06c8f0 0x7fa8cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.912+0000 7fa8deffd640 1 -- 172.21.15.50:0/3928875447 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fa8d80ada80 con 0x7fa8e80b8da0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.912+0000 7fa8ee1f1640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa8cc06c8f0 0x7fa8cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.913+0000 7fa8ee1f1640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa8cc06c8f0 0x7fa8cc06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009b10 tx=0x7fa8e4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:46.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:46.916+0000 7fa8deffd640 1 -- 172.21.15.50:0/3928875447 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa8d804b030 con 0x7fa8e80b8da0 2024-04-02T16:22:47.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.103+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa8e80c7550 con 0x7fa8e80b8da0 2024-04-02T16:22:47.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.104+0000 7fa8deffd640 1 -- 172.21.15.50:0/3928875447 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa8d807ba00 con 0x7fa8e80b8da0 2024-04-02T16:22:47.104 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:47.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa8cc06c8f0 msgr2=0x7fa8cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa8cc06c8f0 0x7fa8cc06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009b10 tx=0x7fa8e4008040 comp rx=0 tx=0).stop 2024-04-02T16:22:47.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e80b8da0 msgr2=0x7fa8e80b9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e80b8da0 0x7fa8e80b9220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8000c00 tx=0x7fa8d8001120 comp rx=0 tx=0).stop 2024-04-02T16:22:47.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 shutdown_connections 2024-04-02T16:22:47.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8e815ed50 0x7fa8e815f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa8cc06c8f0 0x7fa8cc06edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8e80b8da0 0x7fa8e80b9220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 --2- 172.21.15.50:0/3928875447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d4004800 0x7fa8e814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 >> 172.21.15.50:0/3928875447 conn(0x7fa8e808dc90 msgr2=0x7fa8e815c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:47.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 shutdown_connections 2024-04-02T16:22:47.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.106+0000 7fa8f0455640 1 -- 172.21.15.50:0/3928875447 wait complete. 2024-04-02T16:22:47.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:47.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18ee77640 1 Processor -- start 2024-04-02T16:22:47.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18ee77640 1 -- start start 2024-04-02T16:22:47.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18ee77640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa188145c60 0x7fa18806b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18ee77640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa1880ce1a0 0x7fa18806ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18ee77640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1880cfdc0 con 0x7fa1880d0b80 2024-04-02T16:22:47.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18ee77640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fa188146040 con 0x7fa1880ce1a0 2024-04-02T16:22:47.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18ee77640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa1880d00a0 con 0x7fa188145c60 2024-04-02T16:22:47.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.221+0000 7fa18d414640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa1880ce1a0 0x7fa18806ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.222+0000 7fa18d414640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa1880ce1a0 0x7fa18806ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37974/0 (socket says 172.21.15.50:37974) 2024-04-02T16:22:47.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.222+0000 7fa18d414640 1 -- 172.21.15.50:0/127257494 learned_addr learned my addr 172.21.15.50:0/127257494 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:47.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.222+0000 7fa18d414640 1 -- 172.21.15.50:0/127257494 >> v2:172.21.15.188:3300/0 conn(0x7fa188145c60 msgr2=0x7fa18806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:47.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.222+0000 7fa18cc13640 1 --2- 172.21.15.50:0/127257494 >> v2:172.21.15.188:3300/0 conn(0x7fa188145c60 0x7fa18806b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.222+0000 7fa18d414640 1 --2- 172.21.15.50:0/127257494 >> v2:172.21.15.188:3300/0 conn(0x7fa188145c60 0x7fa18806b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.222+0000 7fa18d414640 1 -- 172.21.15.50:0/127257494 >> v1:172.21.15.188:6789/0 conn(0x7fa1880d0b80 legacy=0x7fa1880cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:47.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.222+0000 7fa18d414640 1 -- 172.21.15.50:0/127257494 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1880d0380 con 0x7fa1880ce1a0 2024-04-02T16:22:47.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18d414640 1 --2- 172.21.15.50:0/127257494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1880ce1a0 0x7fa18806ba60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa1880d2140 tx=0x7fa18002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f73ed166a0bcb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa1877fe640 1 -- 172.21.15.50:0/127257494 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18002c9f0 con 0x7fa1880ce1a0 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa1877fe640 1 -- 172.21.15.50:0/127257494 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18002cb90 con 0x7fa1880ce1a0 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa1877fe640 1 -- 172.21.15.50:0/127257494 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1800343f0 con 0x7fa1880ce1a0 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 -- 172.21.15.50:0/127257494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1880ce1a0 msgr2=0x7fa18806ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 --2- 172.21.15.50:0/127257494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1880ce1a0 0x7fa18806ba60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa1880d2140 tx=0x7fa18002be50 comp rx=0 tx=0).stop 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 -- 172.21.15.50:0/127257494 shutdown_connections 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 --2- 172.21.15.50:0/127257494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1880ce1a0 0x7fa18806ba60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 --2- 172.21.15.50:0/127257494 >> v2:172.21.15.188:3300/0 conn(0x7fa188145c60 0x7fa18806b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 -- 172.21.15.50:0/127257494 >> 172.21.15.50:0/127257494 conn(0x7fa1880b2db0 msgr2=0x7fa1880b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 -- 172.21.15.50:0/127257494 shutdown_connections 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.223+0000 7fa18ee77640 1 -- 172.21.15.50:0/127257494 wait complete. 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 Processor -- start 2024-04-02T16:22:47.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 -- start start 2024-04-02T16:22:47.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1880ce1a0 0x7fa188142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1880d0b80 0x7fa188151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa188145c60 0x7fa1881f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa18815c890 con 0x7fa188145c60 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880cfad0 con 0x7fa1880d0b80 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.225+0000 7fa18ee77640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa18815ca10 con 0x7fa1880ce1a0 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa188145c60 0x7fa1881f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa188145c60 0x7fa1881f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37978/0 (socket says 172.21.15.50:37978) 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 -- 172.21.15.50:0/2003029564 learned_addr learned my addr 172.21.15.50:0/2003029564 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18cc13640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1880ce1a0 0x7fa188142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa187fff640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1880d0b80 0x7fa188151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 -- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1880ce1a0 msgr2=0x7fa188142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1880ce1a0 0x7fa188142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 -- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1880d0b80 msgr2=0x7fa188151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1880d0b80 0x7fa188151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 -- 172.21.15.50:0/2003029564 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa188144000 con 0x7fa188145c60 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18d414640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa188145c60 0x7fa1881f2130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa180036040 tx=0x7fa180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa1857fa640 1 -- 172.21.15.50:0/2003029564 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180043020 con 0x7fa188145c60 2024-04-02T16:22:47.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa1857fa640 1 -- 172.21.15.50:0/2003029564 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18003e450 con 0x7fa188145c60 2024-04-02T16:22:47.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa188143d20 con 0x7fa188145c60 2024-04-02T16:22:47.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa1857fa640 1 -- 172.21.15.50:0/2003029564 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18003e820 con 0x7fa188145c60 2024-04-02T16:22:47.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.226+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa188141e00 con 0x7fa188145c60 2024-04-02T16:22:47.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.227+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa188075760 con 0x7fa188145c60 2024-04-02T16:22:47.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.228+0000 7fa1857fa640 1 -- 172.21.15.50:0/2003029564 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa180046020 con 0x7fa188145c60 2024-04-02T16:22:47.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.228+0000 7fa1857fa640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa16c06c8f0 0x7fa16c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.228+0000 7fa1857fa640 1 -- 172.21.15.50:0/2003029564 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fa1800ae7f0 con 0x7fa188145c60 2024-04-02T16:22:47.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.228+0000 7fa18cc13640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa16c06c8f0 0x7fa16c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.229+0000 7fa18cc13640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa16c06c8f0 0x7fa16c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa188070b30 tx=0x7fa17c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.233+0000 7fa1857fa640 1 -- 172.21.15.50:0/2003029564 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa18004d030 con 0x7fa188145c60 2024-04-02T16:22:47.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.417+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa188068840 con 0x7fa188145c60 2024-04-02T16:22:47.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.417+0000 7fa1857fa640 1 -- 172.21.15.50:0/2003029564 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fa18007c1b0 con 0x7fa188145c60 2024-04-02T16:22:47.418 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa16c06c8f0 msgr2=0x7fa16c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa16c06c8f0 0x7fa16c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa188070b30 tx=0x7fa17c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa188145c60 msgr2=0x7fa1881f2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa188145c60 0x7fa1881f2130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa180036040 tx=0x7fa180004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 shutdown_connections 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa16c06c8f0 0x7fa16c06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa188145c60 0x7fa1881f2130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1880d0b80 0x7fa188151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 --2- 172.21.15.50:0/2003029564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1880ce1a0 0x7fa188142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 >> 172.21.15.50:0/2003029564 conn(0x7fa1880b2db0 msgr2=0x7fa1880d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 shutdown_connections 2024-04-02T16:22:47.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.419+0000 7fa18ee77640 1 -- 172.21.15.50:0/2003029564 wait complete. 2024-04-02T16:22:47.431 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.532+0000 7f5e1193d640 1 Processor -- start 2024-04-02T16:22:47.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.532+0000 7f5e1193d640 1 -- start start 2024-04-02T16:22:47.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e1193d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5e0c15e510 0x7f5e0c0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e1193d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c0d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e1193d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0c0ceda0 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e1193d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f5e0c0d31c0 con 0x7f5e0c15e510 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e1193d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c0cf040 con 0x7f5e0c0d2800 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0affd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5e0c15e510 0x7f5e0c0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0affd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5e0c15e510 0x7f5e0c0cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37986/0 (socket says 172.21.15.50:37986) 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0affd640 1 -- 172.21.15.50:0/77940304 learned_addr learned my addr 172.21.15.50:0/77940304 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0a7fc640 1 --2- 172.21.15.50:0/77940304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c0d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0affd640 1 -- 172.21.15.50:0/77940304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 msgr2=0x7f5e0c0d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0affd640 1 --2- 172.21.15.50:0/77940304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c0d2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0affd640 1 -- 172.21.15.50:0/77940304 >> v1:172.21.15.188:6789/0 conn(0x7f5e0c0d0e90 legacy=0x7f5e0c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.533+0000 7f5e0affd640 1 -- 172.21.15.50:0/77940304 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c0cf320 con 0x7f5e0c15e510 2024-04-02T16:22:47.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.534+0000 7f5e0affd640 1 --2- 172.21.15.50:0/77940304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c15e510 0x7f5e0c0cfe00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009870 tx=0x7f5e0402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c479a7c90d3121 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.534+0000 7f5e09ffb640 1 -- 172.21.15.50:0/77940304 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04035020 con 0x7f5e0c15e510 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.534+0000 7f5e09ffb640 1 -- 172.21.15.50:0/77940304 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e04031640 con 0x7f5e0c15e510 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.534+0000 7f5e09ffb640 1 -- 172.21.15.50:0/77940304 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0402d040 con 0x7f5e0c15e510 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 -- 172.21.15.50:0/77940304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c15e510 msgr2=0x7f5e0c0cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 --2- 172.21.15.50:0/77940304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c15e510 0x7f5e0c0cfe00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009870 tx=0x7f5e0402b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 -- 172.21.15.50:0/77940304 shutdown_connections 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 --2- 172.21.15.50:0/77940304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c0d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 --2- 172.21.15.50:0/77940304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c15e510 0x7f5e0c0cfe00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 -- 172.21.15.50:0/77940304 >> 172.21.15.50:0/77940304 conn(0x7f5e0c09d570 msgr2=0x7f5e0c0d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 -- 172.21.15.50:0/77940304 shutdown_connections 2024-04-02T16:22:47.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.535+0000 7f5e1193d640 1 -- 172.21.15.50:0/77940304 wait complete. 2024-04-02T16:22:47.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.536+0000 7f5e1193d640 1 Processor -- start 2024-04-02T16:22:47.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.536+0000 7f5e1193d640 1 -- start start 2024-04-02T16:22:47.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e1193d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c0d0e90 0x7f5e0c0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e1193d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e1193d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c15e510 0x7f5e0c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e1193d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c0b3ad0 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e1193d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c15e8f0 con 0x7f5e0c15e510 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e1193d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c0b3c50 con 0x7f5e0c0d2800 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c0d0e90 0x7f5e0c0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c0d0e90 0x7f5e0c0ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37988/0 (socket says 172.21.15.50:37988) 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 -- 172.21.15.50:0/4160587061 learned_addr learned my addr 172.21.15.50:0/4160587061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0b7fe640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c15e510 0x7f5e0c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0a7fc640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 -- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 msgr2=0x7f5e0c152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 -- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c15e510 msgr2=0x7f5e0c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c15e510 0x7f5e0c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 -- 172.21.15.50:0/4160587061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1595d0 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0b7fe640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c15e510 0x7f5e0c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0a7fc640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:47.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.537+0000 7f5e0affd640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c0d0e90 0x7f5e0c0ca750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5e04000c00 tx=0x7f5e040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.538+0000 7f5e03fff640 1 -- 172.21.15.50:0/4160587061 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04040070 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.538+0000 7f5e03fff640 1 -- 172.21.15.50:0/4160587061 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e040399f0 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.538+0000 7f5e03fff640 1 -- 172.21.15.50:0/4160587061 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04035040 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.538+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c1598b0 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.538+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c159e70 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.539+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c0756f0 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.539+0000 7f5e03fff640 1 -- 172.21.15.50:0/4160587061 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e0404b060 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.540+0000 7f5e03fff640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5dec06c8f0 0x7f5dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.540+0000 7f5e03fff640 1 -- 172.21.15.50:0/4160587061 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f5e040ad750 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.540+0000 7f5e0a7fc640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5dec06c8f0 0x7f5dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.540+0000 7f5e0a7fc640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5dec06c8f0 0x7f5dec06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ce790 tx=0x7f5df4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.544+0000 7f5e03fff640 1 -- 172.21.15.50:0/4160587061 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e0407b760 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.732+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e0c160200 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.733+0000 7f5e03fff640 1 -- 172.21.15.50:0/4160587061 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7f5e0407b060 con 0x7f5e0c0d0e90 2024-04-02T16:22:47.734 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:47.734 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.735+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5dec06c8f0 msgr2=0x7f5dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.735+0000 7f5e1193d640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5dec06c8f0 0x7f5dec06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ce790 tx=0x7f5df4008040 comp rx=0 tx=0).stop 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.735+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c0d0e90 msgr2=0x7f5e0c0ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.735+0000 7f5e1193d640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c0d0e90 0x7f5e0c0ca750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5e04000c00 tx=0x7f5e040046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 shutdown_connections 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c15e510 0x7f5e0c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e0c0d2800 0x7f5e0c152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5dec06c8f0 0x7f5dec06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 --2- 172.21.15.50:0/4160587061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e0c0d0e90 0x7f5e0c0ca750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 >> 172.21.15.50:0/4160587061 conn(0x7f5e0c09d570 msgr2=0x7f5e0c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:47.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 shutdown_connections 2024-04-02T16:22:47.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.736+0000 7f5e1193d640 1 -- 172.21.15.50:0/4160587061 wait complete. 2024-04-02T16:22:47.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:47.747 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:47.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.851+0000 7f948542a640 1 Processor -- start 2024-04-02T16:22:47.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.851+0000 7f948542a640 1 -- start start 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f948542a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f94800b4b50 0x7f948015f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f948542a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f94800b2cb0 0x7f94800b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f948542a640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9480162a10 con 0x7f94800b4770 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f948542a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f94800b3660 con 0x7f94800b4b50 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f948542a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9480162cf0 con 0x7f94800b2cb0 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f947e7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f94800b4b50 0x7f948015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f947effd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f94800b4770 0x7f9480161f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34628/0 (socket says 172.21.15.50:34628) 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.852+0000 7f947effd640 1 -- 172.21.15.50:0/3650740771 learned_addr learned my addr 172.21.15.50:0/3650740771 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:47.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947f7fe640 1 --2- 172.21.15.50:0/3650740771 >> v2:172.21.15.188:3300/0 conn(0x7f94800b2cb0 0x7f94800b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947e7fc640 1 -- 172.21.15.50:0/3650740771 >> v2:172.21.15.188:3300/0 conn(0x7f94800b2cb0 msgr2=0x7f94800b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:47.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947e7fc640 1 --2- 172.21.15.50:0/3650740771 >> v2:172.21.15.188:3300/0 conn(0x7f94800b2cb0 0x7f94800b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947e7fc640 1 -- 172.21.15.50:0/3650740771 >> v1:172.21.15.50:6789/0 conn(0x7f94800b4770 legacy=0x7f9480161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947e7fc640 1 -- 172.21.15.50:0/3650740771 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948015ec70 con 0x7f94800b4b50 2024-04-02T16:22:47.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947dffb640 1 -- 172.21.15.50:0/3650740771 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3842836533 0 0) 0x7f9480162a10 con 0x7f94800b4770 2024-04-02T16:22:47.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947f7fe640 1 --2- 172.21.15.50:0/3650740771 >> v2:172.21.15.188:3300/0 conn(0x7f94800b2cb0 0x7f94800b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:47.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947e7fc640 1 --2- 172.21.15.50:0/3650740771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f948015f470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94800b3de0 tx=0x7f947402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8248e63ff6e79faf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.853+0000 7f947dffb640 1 -- 172.21.15.50:0/3650740771 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474035020 con 0x7f94800b4b50 2024-04-02T16:22:47.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f947dffb640 1 -- 172.21.15.50:0/3650740771 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9474031830 con 0x7f94800b4b50 2024-04-02T16:22:47.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f947dffb640 1 -- 172.21.15.50:0/3650740771 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474031c00 con 0x7f94800b4b50 2024-04-02T16:22:47.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 -- 172.21.15.50:0/3650740771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 msgr2=0x7f948015f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 --2- 172.21.15.50:0/3650740771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f948015f470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94800b3de0 tx=0x7f947402b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 -- 172.21.15.50:0/3650740771 shutdown_connections 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 --2- 172.21.15.50:0/3650740771 >> v2:172.21.15.188:3300/0 conn(0x7f94800b2cb0 0x7f94800b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 --2- 172.21.15.50:0/3650740771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f948015f470 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 -- 172.21.15.50:0/3650740771 >> 172.21.15.50:0/3650740771 conn(0x7f948006e6c0 msgr2=0x7f948006eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 -- 172.21.15.50:0/3650740771 shutdown_connections 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.854+0000 7f948542a640 1 -- 172.21.15.50:0/3650740771 wait complete. 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.855+0000 7f948542a640 1 Processor -- start 2024-04-02T16:22:47.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.855+0000 7f948542a640 1 -- start start 2024-04-02T16:22:47.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f948542a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800b2cb0 0x7f94800afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f948542a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b4770 0x7f9480158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f948542a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f94801f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f948542a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9480073080 con 0x7f94800b4b50 2024-04-02T16:22:47.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f948542a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9480072f00 con 0x7f94800b2cb0 2024-04-02T16:22:47.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f948542a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9480162660 con 0x7f94800b4770 2024-04-02T16:22:47.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f94801f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f94801f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37994/0 (socket says 172.21.15.50:37994) 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f947f7fe640 1 -- 172.21.15.50:0/2046082483 learned_addr learned my addr 172.21.15.50:0/2046082483 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f947effd640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800b2cb0 0x7f94800afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.856+0000 7f947e7fc640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b4770 0x7f9480158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f947f7fe640 1 -- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b4770 msgr2=0x7f9480158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f947f7fe640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b4770 0x7f9480158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f947f7fe640 1 -- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800b2cb0 msgr2=0x7f94800afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f947f7fe640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800b2cb0 0x7f94800afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f947f7fe640 1 -- 172.21.15.50:0/2046082483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948015c880 con 0x7f94800b4b50 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f947effd640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800b2cb0 0x7f94800afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f947f7fe640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f94801f3170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f947000cf20 tx=0x7f947000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f94677fe640 1 -- 172.21.15.50:0/2046082483 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947001a070 con 0x7f94800b4b50 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f94677fe640 1 -- 172.21.15.50:0/2046082483 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947000d840 con 0x7f94800b4b50 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9480159a90 con 0x7f94800b4b50 2024-04-02T16:22:47.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f94677fe640 1 -- 172.21.15.50:0/2046082483 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947000dc30 con 0x7f94800b4b50 2024-04-02T16:22:47.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.857+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948015cb20 con 0x7f94800b4b50 2024-04-02T16:22:47.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.858+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9480074a10 con 0x7f94800b4b50 2024-04-02T16:22:47.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.858+0000 7f94677fe640 1 -- 172.21.15.50:0/2046082483 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f947001d070 con 0x7f94800b4b50 2024-04-02T16:22:47.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.859+0000 7f94677fe640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f946c06c8f0 0x7f946c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:47.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.859+0000 7f94677fe640 1 -- 172.21.15.50:0/2046082483 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f947008c8b0 con 0x7f94800b4b50 2024-04-02T16:22:47.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.859+0000 7f947effd640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f946c06c8f0 0x7f946c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:47.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.860+0000 7f947effd640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f946c06c8f0 0x7f946c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9478000f60 tx=0x7f9478007c10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:47.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:47.863+0000 7f94677fe640 1 -- 172.21.15.50:0/2046082483 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9480159a90 con 0x7f94800b4b50 2024-04-02T16:22:48.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.046+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9480075760 con 0x7f94800b4b50 2024-04-02T16:22:48.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.046+0000 7f94677fe640 1 -- 172.21.15.50:0/2046082483 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f947005a820 con 0x7f94800b4b50 2024-04-02T16:22:48.047 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f946c06c8f0 msgr2=0x7f946c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f946c06c8f0 0x7f946c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9478000f60 tx=0x7f9478007c10 comp rx=0 tx=0).stop 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 msgr2=0x7f94801f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f94801f3170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f947000cf20 tx=0x7f947000a7d0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 shutdown_connections 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f946c06c8f0 0x7f946c06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94800b4b50 0x7f94801f3170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94800b4770 0x7f9480158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 --2- 172.21.15.50:0/2046082483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94800b2cb0 0x7f94800afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 >> 172.21.15.50:0/2046082483 conn(0x7f948006e6c0 msgr2=0x7f94800b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 shutdown_connections 2024-04-02T16:22:48.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.048+0000 7f948542a640 1 -- 172.21.15.50:0/2046082483 wait complete. 2024-04-02T16:22:48.061 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:48.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.164+0000 7fd6789af640 1 Processor -- start 2024-04-02T16:22:48.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.164+0000 7fd6789af640 1 -- start start 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd6789af640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd674144bd0 0x7fd674072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd6789af640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd6740b58b0 0x7fd6740b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd6789af640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd674073350 con 0x7fd6740b31f0 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd6789af640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fd6740ce760 con 0x7fd6740b58b0 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd6789af640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd674072880 con 0x7fd674144bd0 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd66bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd6740b58b0 0x7fd6740b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd66bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd6740b58b0 0x7fd6740b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38004/0 (socket says 172.21.15.50:38004) 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.165+0000 7fd66bfff640 1 -- 172.21.15.50:0/3808776121 learned_addr learned my addr 172.21.15.50:0/3808776121 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.166+0000 7fd66bfff640 1 -- 172.21.15.50:0/3808776121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd674144bd0 msgr2=0x7fd674072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.166+0000 7fd66bfff640 1 --2- 172.21.15.50:0/3808776121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd674144bd0 0x7fd674072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.166+0000 7fd66bfff640 1 -- 172.21.15.50:0/3808776121 >> v1:172.21.15.188:6789/0 conn(0x7fd6740b31f0 legacy=0x7fd67406ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.166+0000 7fd66bfff640 1 -- 172.21.15.50:0/3808776121 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd674072ab0 con 0x7fd6740b58b0 2024-04-02T16:22:48.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.166+0000 7fd66bfff640 1 --2- 172.21.15.50:0/3808776121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b58b0 0x7fd6740b2cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009870 tx=0x7fd65c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8575175cfecfac52 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd671d9b640 1 -- 172.21.15.50:0/3808776121 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c035020 con 0x7fd6740b58b0 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd671d9b640 1 -- 172.21.15.50:0/3808776121 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c031640 con 0x7fd6740b58b0 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd671d9b640 1 -- 172.21.15.50:0/3808776121 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c02d040 con 0x7fd6740b58b0 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 -- 172.21.15.50:0/3808776121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b58b0 msgr2=0x7fd6740b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 --2- 172.21.15.50:0/3808776121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b58b0 0x7fd6740b2cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009870 tx=0x7fd65c02b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 -- 172.21.15.50:0/3808776121 shutdown_connections 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 --2- 172.21.15.50:0/3808776121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b58b0 0x7fd6740b2cb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 --2- 172.21.15.50:0/3808776121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd674144bd0 0x7fd674072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 -- 172.21.15.50:0/3808776121 >> 172.21.15.50:0/3808776121 conn(0x7fd67406e6c0 msgr2=0x7fd67406eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 -- 172.21.15.50:0/3808776121 shutdown_connections 2024-04-02T16:22:48.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.167+0000 7fd6789af640 1 -- 172.21.15.50:0/3808776121 wait complete. 2024-04-02T16:22:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.168+0000 7fd6789af640 1 Processor -- start 2024-04-02T16:22:48.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd6789af640 1 -- start start 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd6789af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b31f0 0x7fd674140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd6789af640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6740b58b0 0x7fd6741f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd6789af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd674144bd0 0x7fd6741e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd6789af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740ce990 con 0x7fd6740b31f0 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd6789af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd674073070 con 0x7fd674144bd0 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd6789af640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd6740b5d10 con 0x7fd6740b58b0 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd67259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b31f0 0x7fd674140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd67259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b31f0 0x7fd674140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38014/0 (socket says 172.21.15.50:38014) 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd67259c640 1 -- 172.21.15.50:0/2168292753 learned_addr learned my addr 172.21.15.50:0/2168292753 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd66bfff640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6740b58b0 0x7fd6741f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.169+0000 7fd672d9d640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd674144bd0 0x7fd6741e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd67259c640 1 -- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6740b58b0 msgr2=0x7fd6741f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd67259c640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6740b58b0 0x7fd6741f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd67259c640 1 -- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd674144bd0 msgr2=0x7fd6741e38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd67259c640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd674144bd0 0x7fd6741e38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd67259c640 1 -- 172.21.15.50:0/2168292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6741f5f90 con 0x7fd6740b31f0 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd66bfff640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6740b58b0 0x7fd6741f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd672d9d640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd674144bd0 0x7fd6741e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd67259c640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b31f0 0x7fd674140c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b410 tx=0x7fd66400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd66affd640 1 -- 172.21.15.50:0/2168292753 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664015020 con 0x7fd6740b31f0 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd66affd640 1 -- 172.21.15.50:0/2168292753 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd664011850 con 0x7fd6740b31f0 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6741f2430 con 0x7fd6740b31f0 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd66affd640 1 -- 172.21.15.50:0/2168292753 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664011c40 con 0x7fd6740b31f0 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.170+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6741f6510 con 0x7fd6740b31f0 2024-04-02T16:22:48.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.171+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd674074a10 con 0x7fd6740b31f0 2024-04-02T16:22:48.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.171+0000 7fd66affd640 1 -- 172.21.15.50:0/2168292753 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd664018020 con 0x7fd6740b31f0 2024-04-02T16:22:48.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.172+0000 7fd66affd640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.172+0000 7fd66affd640 1 -- 172.21.15.50:0/2168292753 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fd66408bea0 con 0x7fd6740b31f0 2024-04-02T16:22:48.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.172+0000 7fd66bfff640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.173+0000 7fd66bfff640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd65406c8f0 0x7fd65406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd65c036000 tx=0x7fd65c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.176+0000 7fd66affd640 1 -- 172.21.15.50:0/2168292753 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd66401f020 con 0x7fd6740b31f0 2024-04-02T16:22:48.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.362+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6740b3920 con 0x7fd6740b31f0 2024-04-02T16:22:48.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.362+0000 7fd66affd640 1 -- 172.21.15.50:0/2168292753 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fd6640172c0 con 0x7fd6740b31f0 2024-04-02T16:22:48.364 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:48.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd65406c8f0 msgr2=0x7fd65406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd65406c8f0 0x7fd65406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd65c036000 tx=0x7fd65c03c040 comp rx=0 tx=0).stop 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b31f0 msgr2=0x7fd674140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b31f0 0x7fd674140c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b410 tx=0x7fd66400b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 shutdown_connections 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd674144bd0 0x7fd6741e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6740b58b0 0x7fd6741f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd65406c8f0 0x7fd65406edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 --2- 172.21.15.50:0/2168292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6740b31f0 0x7fd674140c20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 >> 172.21.15.50:0/2168292753 conn(0x7fd67406e6c0 msgr2=0x7fd6740cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 shutdown_connections 2024-04-02T16:22:48.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.364+0000 7fd6789af640 1 -- 172.21.15.50:0/2168292753 wait complete. 2024-04-02T16:22:48.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:48.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.480+0000 7ffa8f31c640 1 Processor -- start 2024-04-02T16:22:48.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.480+0000 7ffa8f31c640 1 -- start start 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa8f31c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffa880d4980 0x7ffa88162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa8f31c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa880d5000 con 0x7ffa880d0c80 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa8f31c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa880cf0d0 con 0x7ffa88161ca0 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa8f31c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffa881630f0 con 0x7ffa880d4980 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa8d8b9640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ffa880d0c80 0x7ffa880d1090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34632/0 (socket says 172.21.15.50:34632) 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa8d8b9640 1 -- 172.21.15.50:0/1155940527 learned_addr learned my addr 172.21.15.50:0/1155940527 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa8c8b7640 1 --2- 172.21.15.50:0/1155940527 >> v2:172.21.15.188:3300/0 conn(0x7ffa880d4980 0x7ffa88162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649009124 0 0) 0x7ffa880d5000 con 0x7ffa880d0c80 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa74003520 con 0x7ffa880d0c80 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3187017337 0 0) 0x7ffa74003520 con 0x7ffa880d0c80 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 >> v2:172.21.15.188:3300/0 conn(0x7ffa880d4980 msgr2=0x7ffa88162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 --2- 172.21.15.50:0/1155940527 >> v2:172.21.15.188:3300/0 conn(0x7ffa880d4980 0x7ffa88162bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 >> v1:172.21.15.188:6789/0 conn(0x7ffa88161ca0 legacy=0x7ffa880cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa880cf3b0 con 0x7ffa880d0c80 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7ffa80001a10 con 0x7ffa880d0c80 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.481+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 >> v1:172.21.15.50:6789/0 conn(0x7ffa880d0c80 legacy=0x7ffa880d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa7ffff640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa7ffff640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa74004800 0x7ffa74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa7ffff640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa740052e0 0x7ffa7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881630f0 con 0x7ffa740039b0 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7400c1b0 con 0x7ffa74004800 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7400c3b0 con 0x7ffa740052e0 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d0b8640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8c8b7640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa74004800 0x7ffa74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d8b9640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa740052e0 0x7ffa7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d0b8640 1 -- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa740052e0 msgr2=0x7ffa7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d0b8640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa740052e0 0x7ffa7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d0b8640 1 -- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa74004800 msgr2=0x7ffa74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d0b8640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa74004800 0x7ffa74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d8b9640 1 -- 172.21.15.50:0/1155940527 reap_dead start 2024-04-02T16:22:48.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8d0b8640 1 -- 172.21.15.50:0/1155940527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa74003520 con 0x7ffa740039b0 2024-04-02T16:22:48.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.482+0000 7ffa8c8b7640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa74004800 0x7ffa74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:48.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8d0b8640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa74003dd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa84001720 tx=0x7ffa74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c11b3fecc3afa29 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84001370 con 0x7ffa740039b0 2024-04-02T16:22:48.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa840052d0 con 0x7ffa740039b0 2024-04-02T16:22:48.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa7ffff640 1 -- 172.21.15.50:0/1155940527 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84003830 con 0x7ffa740039b0 2024-04-02T16:22:48.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8f31c640 1 -- 172.21.15.50:0/1155940527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 msgr2=0x7ffa74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa74003dd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa84001720 tx=0x7ffa74005830 comp rx=0 tx=0).stop 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8f31c640 1 -- 172.21.15.50:0/1155940527 shutdown_connections 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/1155940527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa74003dd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8f31c640 1 -- 172.21.15.50:0/1155940527 >> 172.21.15.50:0/1155940527 conn(0x7ffa8809d5c0 msgr2=0x7ffa880d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8f31c640 1 -- 172.21.15.50:0/1155940527 shutdown_connections 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.483+0000 7ffa8f31c640 1 -- 172.21.15.50:0/1155940527 wait complete. 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 Processor -- start 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 -- start start 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa881547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa880ccad0 0x7ffa880ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa880b9580 0x7ffa880b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffa88162bb0 con 0x7ffa740039b0 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa880ced80 con 0x7ffa880ccad0 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8f31c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffa88162d30 con 0x7ffa880b9580 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8d0b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa881547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8d0b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa881547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38030/0 (socket says 172.21.15.50:38030) 2024-04-02T16:22:48.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.485+0000 7ffa8d0b8640 1 -- 172.21.15.50:0/3766292468 learned_addr learned my addr 172.21.15.50:0/3766292468 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8d8b9640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa880b9580 0x7ffa880b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8c8b7640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa880ccad0 0x7ffa880ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8d0b8640 1 -- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa880b9580 msgr2=0x7ffa880b9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8d0b8640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa880b9580 0x7ffa880b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8d0b8640 1 -- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa880ccad0 msgr2=0x7ffa880ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8d0b8640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa880ccad0 0x7ffa880ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8d0b8640 1 -- 172.21.15.50:0/3766292468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa880b9f40 con 0x7ffa740039b0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8c8b7640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa880ccad0 0x7ffa880ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8d0b8640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa881547a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa84001840 tx=0x7ffa84001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa7dffb640 1 -- 172.21.15.50:0/3766292468 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84041070 con 0x7ffa740039b0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa7dffb640 1 -- 172.21.15.50:0/3766292468 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa84005480 con 0x7ffa740039b0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa88154ec0 con 0x7ffa740039b0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa7dffb640 1 -- 172.21.15.50:0/3766292468 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84007a30 con 0x7ffa740039b0 2024-04-02T16:22:48.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.486+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa880cd640 con 0x7ffa740039b0 2024-04-02T16:22:48.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.487+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa88068f00 con 0x7ffa740039b0 2024-04-02T16:22:48.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.488+0000 7ffa7dffb640 1 -- 172.21.15.50:0/3766292468 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa84044070 con 0x7ffa740039b0 2024-04-02T16:22:48.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.488+0000 7ffa7dffb640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffa6c06c920 0x7ffa6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.488+0000 7ffa7dffb640 1 -- 172.21.15.50:0/3766292468 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ffa840ada50 con 0x7ffa740039b0 2024-04-02T16:22:48.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.489+0000 7ffa8c8b7640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffa6c06c920 0x7ffa6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.489+0000 7ffa8c8b7640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffa6c06c920 0x7ffa6c06ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa88068e20 tx=0x7ffa78008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.493+0000 7ffa7dffb640 1 -- 172.21.15.50:0/3766292468 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffa88068f00 con 0x7ffa740039b0 2024-04-02T16:22:48.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.675+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa880d36d0 con 0x7ffa740039b0 2024-04-02T16:22:48.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.675+0000 7ffa7dffb640 1 -- 172.21.15.50:0/3766292468 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ffa880d36d0 con 0x7ffa740039b0 2024-04-02T16:22:48.677 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffa6c06c920 msgr2=0x7ffa6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffa6c06c920 0x7ffa6c06ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa88068e20 tx=0x7ffa78008040 comp rx=0 tx=0).stop 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 msgr2=0x7ffa881547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa881547a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa84001840 tx=0x7ffa84001120 comp rx=0 tx=0).stop 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 shutdown_connections 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa880b9580 0x7ffa880b9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa880ccad0 0x7ffa880ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffa6c06c920 0x7ffa6c06ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 --2- 172.21.15.50:0/3766292468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa740039b0 0x7ffa881547a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 >> 172.21.15.50:0/3766292468 conn(0x7ffa8809d5c0 msgr2=0x7ffa880d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:48.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 shutdown_connections 2024-04-02T16:22:48.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.678+0000 7ffa8f31c640 1 -- 172.21.15.50:0/3766292468 wait complete. 2024-04-02T16:22:48.690 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:48.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.795+0000 7fd87425e640 1 Processor -- start 2024-04-02T16:22:48.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.796+0000 7fd87425e640 1 -- start start 2024-04-02T16:22:48.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.796+0000 7fd87425e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 0x7fd86c0d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.796+0000 7fd87425e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86c0d2f30 con 0x7fd86c15e7b0 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.796+0000 7fd87425e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86c0cf000 con 0x7fd86c0c7e30 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.796+0000 7fd87425e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c0cf2e0 con 0x7fd86c15e3d0 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd8727fb640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd86c15e7b0 0x7fd86c0cfae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34636/0 (socket says 172.21.15.50:34636) 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd8727fb640 1 -- 172.21.15.50:0/4186587782 learned_addr learned my addr 172.21.15.50:0/4186587782 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd871ffa640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 0x7fd86c0d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262691714 0 0) 0x7fd86c0d2f30 con 0x7fd86c15e7b0 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd858003520 con 0x7fd86c15e7b0 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 395786675 0 0) 0x7fd86c0cf000 con 0x7fd86c0c7e30 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd86c0d2f30 con 0x7fd86c0c7e30 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3741645328 0 0) 0x7fd858003520 con 0x7fd86c15e7b0 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 msgr2=0x7fd86c0d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 0x7fd86c0d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 >> v1:172.21.15.188:6789/0 conn(0x7fd86c0c7e30 legacy=0x7fd86c0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c0cf5c0 con 0x7fd86c15e7b0 2024-04-02T16:22:48.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fd8640019f0 con 0x7fd86c15e7b0 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 >> v1:172.21.15.50:6789/0 conn(0x7fd86c15e7b0 legacy=0x7fd86c0cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd858003b70 0x7fd858003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd858004800 0x7fd858004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 0x7fd8580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c0cf2e0 con 0x7fd858003b70 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd85800ba70 con 0x7fd858004800 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd85800bc40 con 0x7fd86c15e3d0 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.797+0000 7fd871ffa640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd858003b70 0x7fd858003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd8717f9640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd858004800 0x7fd858004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd8727fb640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 0x7fd8580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd871ffa640 1 -- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 msgr2=0x7fd8580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd871ffa640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c15e3d0 0x7fd8580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd871ffa640 1 -- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd858004800 msgr2=0x7fd858004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:48.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd871ffa640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd858004800 0x7fd858004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd871ffa640 1 -- 172.21.15.50:0/4186587782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd858003520 con 0x7fd858003b70 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd871ffa640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd858003b70 0x7fd858003f70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd868009ac0 tx=0x7fd858005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d100b1d5c7b0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.798+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868039070 con 0x7fd858003b70 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86802c660 con 0x7fd858003b70 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd870ff8640 1 -- 172.21.15.50:0/4186587782 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868009690 con 0x7fd858003b70 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd87425e640 1 -- 172.21.15.50:0/4186587782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd858003b70 msgr2=0x7fd858003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd87425e640 1 --2- 172.21.15.50:0/4186587782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd858003b70 0x7fd858003f70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd868009ac0 tx=0x7fd858005830 comp rx=0 tx=0).stop 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd8727fb640 1 -- 172.21.15.50:0/4186587782 reap_dead start 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd87425e640 1 -- 172.21.15.50:0/4186587782 shutdown_connections 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd87425e640 1 -- 172.21.15.50:0/4186587782 >> 172.21.15.50:0/4186587782 conn(0x7fd86c08dc90 msgr2=0x7fd86c09d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd87425e640 1 -- 172.21.15.50:0/4186587782 shutdown_connections 2024-04-02T16:22:48.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.799+0000 7fd87425e640 1 -- 172.21.15.50:0/4186587782 wait complete. 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.800+0000 7fd87425e640 1 Processor -- start 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.800+0000 7fd87425e640 1 -- start start 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd87425e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd858003b70 0x7fd86c160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd87425e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd86c15cd50 0x7fd86c1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd87425e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd86c15d1b0 0x7fd86c1f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd87425e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c0d2a00 con 0x7fd86c15cd50 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd87425e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c0d2880 con 0x7fd86c15d1b0 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd87425e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c0d2b80 con 0x7fd858003b70 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd86c15cd50 0x7fd86c1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd86c15cd50 0x7fd86c1611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38042/0 (socket says 172.21.15.50:38042) 2024-04-02T16:22:48.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 -- 172.21.15.50:0/1249448291 learned_addr learned my addr 172.21.15.50:0/1249448291 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd871ffa640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd858003b70 0x7fd86c160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8727fb640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd86c15d1b0 0x7fd86c1f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 -- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd858003b70 msgr2=0x7fd86c160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd858003b70 0x7fd86c160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 -- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd86c15d1b0 msgr2=0x7fd86c1f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd86c15d1b0 0x7fd86c1f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8717f9640 1 -- 172.21.15.50:0/1249448291 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c0cc4a0 con 0x7fd86c15cd50 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.801+0000 7fd8727fb640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd86c15d1b0 0x7fd86c1f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.802+0000 7fd8717f9640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd86c15cd50 0x7fd86c1611a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e7e0 tx=0x7fd85c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.802+0000 7fd8627fc640 1 -- 172.21.15.50:0/1249448291 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0050d0 con 0x7fd86c15cd50 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.802+0000 7fd8627fc640 1 -- 172.21.15.50:0/1249448291 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c018070 con 0x7fd86c15cd50 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.802+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86c0cc780 con 0x7fd86c15cd50 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.802+0000 7fd8627fc640 1 -- 172.21.15.50:0/1249448291 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00c6c0 con 0x7fd86c15cd50 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.802+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86c0ccd40 con 0x7fd86c15cd50 2024-04-02T16:22:48.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.803+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd86c075760 con 0x7fd86c15cd50 2024-04-02T16:22:48.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.803+0000 7fd8627fc640 1 -- 172.21.15.50:0/1249448291 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd85c00c860 con 0x7fd86c15cd50 2024-04-02T16:22:48.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.804+0000 7fd8627fc640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd85006c920 0x7fd85006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:48.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.804+0000 7fd8627fc640 1 -- 172.21.15.50:0/1249448291 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fd85c08bcd0 con 0x7fd86c15cd50 2024-04-02T16:22:48.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.804+0000 7fd871ffa640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd85006c920 0x7fd85006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:48.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.805+0000 7fd871ffa640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd85006c920 0x7fd85006ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd868036000 tx=0x7fd868030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:48.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.808+0000 7fd8627fc640 1 -- 172.21.15.50:0/1249448291 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd85c059ce0 con 0x7fd86c15cd50 2024-04-02T16:22:48.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.995+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd86c0d51d0 con 0x7fd86c15cd50 2024-04-02T16:22:48.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.996+0000 7fd8627fc640 1 -- 172.21.15.50:0/1249448291 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fd85c0595e0 con 0x7fd86c15cd50 2024-04-02T16:22:48.998 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd85006c920 msgr2=0x7fd85006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd85006c920 0x7fd85006ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd868036000 tx=0x7fd868030000 comp rx=0 tx=0).stop 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd86c15cd50 msgr2=0x7fd86c1611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd86c15cd50 0x7fd86c1611a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e7e0 tx=0x7fd85c00eca0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 shutdown_connections 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd86c15d1b0 0x7fd86c1f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd85006c920 0x7fd85006ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd86c15cd50 0x7fd86c1611a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 --2- 172.21.15.50:0/1249448291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd858003b70 0x7fd86c160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 >> 172.21.15.50:0/1249448291 conn(0x7fd86c08dc90 msgr2=0x7fd86c0ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 shutdown_connections 2024-04-02T16:22:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:48.998+0000 7fd87425e640 1 -- 172.21.15.50:0/1249448291 wait complete. 2024-04-02T16:22:49.053 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-02T16:22:49.053 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.154+0000 7ffbc0e0d640 1 Processor -- start 2024-04-02T16:22:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.154+0000 7ffbc0e0d640 1 -- start start 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbc0e0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc1584a0 0x7ffbbc15b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbc0e0d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffbbc06bc30 0x7ffbbc15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbc0e0d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbbc15bc50 con 0x7ffbbc06c090 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbc0e0d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ffbbc15acc0 con 0x7ffbbc06bc30 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbc0e0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffbbc15ed10 con 0x7ffbbc1584a0 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbb9d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffbbc06bc30 0x7ffbbc15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbb9d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ffbbc06bc30 0x7ffbbc15e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38054/0 (socket says 172.21.15.50:38054) 2024-04-02T16:22:49.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3710503964 learned_addr learned my addr 172.21.15.50:0/3710503964 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:49.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbba59c640 1 --2- 172.21.15.50:0/3710503964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc1584a0 0x7ffbbc15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3710503964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc1584a0 msgr2=0x7ffbbc15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:49.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbb9d9b640 1 --2- 172.21.15.50:0/3710503964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc1584a0 0x7ffbbc15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3710503964 >> v1:172.21.15.188:6789/0 conn(0x7ffbbc06c090 legacy=0x7ffbbc0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3710503964 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc15eff0 con 0x7ffbbc06bc30 2024-04-02T16:22:49.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.155+0000 7ffbba59c640 1 --2- 172.21.15.50:0/3710503964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc1584a0 0x7ffbbc15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:49.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.156+0000 7ffbb9d9b640 1 --2- 172.21.15.50:0/3710503964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06bc30 0x7ffbbc15e2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009870 tx=0x7ffbb002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6a9f4a3056a1de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.156+0000 7ffbb959a640 1 -- 172.21.15.50:0/3710503964 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0035020 con 0x7ffbbc06bc30 2024-04-02T16:22:49.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.156+0000 7ffbb959a640 1 -- 172.21.15.50:0/3710503964 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb0031640 con 0x7ffbbc06bc30 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.156+0000 7ffbb959a640 1 -- 172.21.15.50:0/3710503964 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb002d040 con 0x7ffbbc06bc30 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3710503964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06bc30 msgr2=0x7ffbbc15e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3710503964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06bc30 0x7ffbbc15e2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009870 tx=0x7ffbb002b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3710503964 shutdown_connections 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3710503964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06bc30 0x7ffbbc15e2b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3710503964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc1584a0 0x7ffbbc15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3710503964 >> 172.21.15.50:0/3710503964 conn(0x7ffbbc06ebf0 msgr2=0x7ffbbc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3710503964 shutdown_connections 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.157+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3710503964 wait complete. 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.158+0000 7ffbc0e0d640 1 Processor -- start 2024-04-02T16:22:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.158+0000 7ffbc0e0d640 1 -- start start 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbc0e0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc06bc30 0x7ffbbc14c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbc0e0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06c090 0x7ffbbc0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbc0e0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbbc1584a0 0x7ffbbc0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbc0e0d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc0b4a80 con 0x7ffbbc06c090 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbc0e0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc15b1f0 con 0x7ffbbc1584a0 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbc0e0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffbbc158d00 con 0x7ffbbc06bc30 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06c090 0x7ffbbc0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06c090 0x7ffbbc0b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38064/0 (socket says 172.21.15.50:38064) 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3678547861 learned_addr learned my addr 172.21.15.50:0/3678547861 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:49.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbbad9d640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbbc1584a0 0x7ffbbc0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbba59c640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc06bc30 0x7ffbbc14c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc06bc30 msgr2=0x7ffbbc14c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc06bc30 0x7ffbbc14c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbbc1584a0 msgr2=0x7ffbbc0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbbc1584a0 0x7ffbbc0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.159+0000 7ffbb9d9b640 1 -- 172.21.15.50:0/3678547861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc0af320 con 0x7ffbbc06c090 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.160+0000 7ffbba59c640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc06bc30 0x7ffbbc14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.160+0000 7ffbb9d9b640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06c090 0x7ffbbc0b0d20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.160+0000 7ffbaaffd640 1 -- 172.21.15.50:0/3678547861 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0040070 con 0x7ffbbc06c090 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.160+0000 7ffbaaffd640 1 -- 172.21.15.50:0/3678547861 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb00399f0 con 0x7ffbbc06c090 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.160+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbbc158f50 con 0x7ffbbc06c090 2024-04-02T16:22:49.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.160+0000 7ffbaaffd640 1 -- 172.21.15.50:0/3678547861 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0035040 con 0x7ffbbc06c090 2024-04-02T16:22:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.160+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbbc159510 con 0x7ffbbc06c090 2024-04-02T16:22:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.161+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbbc159710 con 0x7ffbbc06c090 2024-04-02T16:22:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.161+0000 7ffbaaffd640 1 -- 172.21.15.50:0/3678547861 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbb004b060 con 0x7ffbbc06c090 2024-04-02T16:22:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.162+0000 7ffbaaffd640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb9c06c8f0 0x7ffb9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.162+0000 7ffbaaffd640 1 -- 172.21.15.50:0/3678547861 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ffbb00ad730 con 0x7ffbbc06c090 2024-04-02T16:22:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.162+0000 7ffbba59c640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb9c06c8f0 0x7ffb9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.163+0000 7ffbba59c640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb9c06c8f0 0x7ffb9c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc15e180 tx=0x7ffbb4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.166+0000 7ffbaaffd640 1 -- 172.21.15.50:0/3678547861 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffbbc159710 con 0x7ffbbc06c090 2024-04-02T16:22:49.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.352+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbbc0756f0 con 0x7ffbbc06c090 2024-04-02T16:22:49.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.353+0000 7ffbaaffd640 1 -- 172.21.15.50:0/3678547861 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7ffbb007b7d0 con 0x7ffbbc06c090 2024-04-02T16:22:49.354 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:49.354 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:49.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb9c06c8f0 msgr2=0x7ffb9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb9c06c8f0 0x7ffb9c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc15e180 tx=0x7ffbb4008040 comp rx=0 tx=0).stop 2024-04-02T16:22:49.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06c090 msgr2=0x7ffbbc0b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06c090 0x7ffbbc0b0d20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb00046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 shutdown_connections 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbbc1584a0 0x7ffbbc0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ffb9c06c8f0 0x7ffb9c06edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc06c090 0x7ffbbc0b0d20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 --2- 172.21.15.50:0/3678547861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbbc06bc30 0x7ffbbc14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 >> 172.21.15.50:0/3678547861 conn(0x7ffbbc06ebf0 msgr2=0x7ffbbc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 shutdown_connections 2024-04-02T16:22:49.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.355+0000 7ffbc0e0d640 1 -- 172.21.15.50:0/3678547861 wait complete. 2024-04-02T16:22:49.367 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:49.368 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:49.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.472+0000 7f9f31dba640 1 Processor -- start 2024-04-02T16:22:49.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.472+0000 7f9f31dba640 1 -- start start 2024-04-02T16:22:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f31dba640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f2c0d0cc0 0x7f9f2c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f31dba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 0x7f9f2c0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f31dba640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2c0b10e0 con 0x7f9f2c161c80 2024-04-02T16:22:49.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f31dba640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2c0d1660 con 0x7f9f2c0d0cc0 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f31dba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c0b1380 con 0x7f9f2c0b1e60 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f2b7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f9f2c161c80 0x7f9f2c0d2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34642/0 (socket says 172.21.15.50:34642) 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f2b7fe640 1 -- 172.21.15.50:0/58289605 learned_addr learned my addr 172.21.15.50:0/58289605 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f2affd640 1 --2- 172.21.15.50:0/58289605 >> v2:172.21.15.188:3300/0 conn(0x7f9f2c0d0cc0 0x7f9f2c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f2bfff640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 0x7f9f2c0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760037642 0 0) 0x7f9f2c0b10e0 con 0x7f9f2c161c80 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.473+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f14003520 con 0x7f9f2c161c80 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 595309984 0 0) 0x7f9f14003520 con 0x7f9f2c161c80 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 msgr2=0x7f9f2c0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 0x7f9f2c0b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 >> v2:172.21.15.188:3300/0 conn(0x7f9f2c0d0cc0 msgr2=0x7f9f2c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:49.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 --2- 172.21.15.50:0/58289605 >> v2:172.21.15.188:3300/0 conn(0x7f9f2c0d0cc0 0x7f9f2c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c0b1660 con 0x7f9f2c161c80 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f9f1c001670 con 0x7f9f2c161c80 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 >> v1:172.21.15.50:6789/0 conn(0x7f9f2c161c80 legacy=0x7f9f2c0d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 0x7f9f140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f14003f30 0x7f9f14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14004d40 0x7f9f140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c0b1380 con 0x7f9f14003f30 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1400b9c0 con 0x7f9f14004d40 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1400bb90 con 0x7f9f2c0b1e60 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2affd640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f14003f30 0x7f9f14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2b7fe640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 0x7f9f140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2bfff640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14004d40 0x7f9f140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2affd640 1 -- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 msgr2=0x7f9f140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:49.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2affd640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c0b1e60 0x7f9f140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2affd640 1 -- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14004d40 msgr2=0x7f9f140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2affd640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14004d40 0x7f9f140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2affd640 1 -- 172.21.15.50:0/58289605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f14003520 con 0x7f9f14003f30 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.474+0000 7f9f2bfff640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14004d40 0x7f9f140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.475+0000 7f9f2affd640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f14003f30 0x7f9f14004800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009af0 tx=0x7f9f14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bec73b9519c213 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.475+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20039070 con 0x7f9f14003f30 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.475+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2002c710 con 0x7f9f14003f30 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.475+0000 7f9f2a7fc640 1 -- 172.21.15.50:0/58289605 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20009690 con 0x7f9f14003f30 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.475+0000 7f9f31dba640 1 -- 172.21.15.50:0/58289605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f14003f30 msgr2=0x7f9f14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.475+0000 7f9f31dba640 1 --2- 172.21.15.50:0/58289605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f14003f30 0x7f9f14004800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009af0 tx=0x7f9f14005830 comp rx=0 tx=0).stop 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.476+0000 7f9f2bfff640 1 -- 172.21.15.50:0/58289605 reap_dead start 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.476+0000 7f9f31dba640 1 -- 172.21.15.50:0/58289605 shutdown_connections 2024-04-02T16:22:49.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.476+0000 7f9f31dba640 1 -- 172.21.15.50:0/58289605 >> 172.21.15.50:0/58289605 conn(0x7f9f2c09d5e0 msgr2=0x7f9f2c0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:49.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.476+0000 7f9f31dba640 1 -- 172.21.15.50:0/58289605 shutdown_connections 2024-04-02T16:22:49.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.476+0000 7f9f31dba640 1 -- 172.21.15.50:0/58289605 wait complete. 2024-04-02T16:22:49.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.477+0000 7f9f31dba640 1 Processor -- start 2024-04-02T16:22:49.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.477+0000 7f9f31dba640 1 -- start start 2024-04-02T16:22:49.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14003f30 0x7f9f2c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c149320 0x7f9f2c15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c15f700 0x7f9f2c0b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c0d0180 con 0x7f9f2c15f700 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c0d3340 con 0x7f9f14003f30 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c0d0300 con 0x7f9f2c149320 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c15f700 0x7f9f2c0b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c15f700 0x7f9f2c0b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38084/0 (socket says 172.21.15.50:38084) 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 -- 172.21.15.50:0/548543730 learned_addr learned my addr 172.21.15.50:0/548543730 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 -- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c149320 msgr2=0x7f9f2c15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c149320 0x7f9f2c15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 -- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14003f30 msgr2=0x7f9f2c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14003f30 0x7f9f2c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 -- 172.21.15.50:0/548543730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c0b9500 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f2bfff640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c15f700 0x7f9f2c0b8010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c08e620 tx=0x7f9f24010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f13fff640 1 -- 172.21.15.50:0/548543730 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24010d80 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f13fff640 1 -- 172.21.15.50:0/548543730 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24003990 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c0cb450 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f13fff640 1 -- 172.21.15.50:0/548543730 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24003d80 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.478+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c0cba10 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.479+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c0cbd20 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.480+0000 7f9f13fff640 1 -- 172.21.15.50:0/548543730 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f24005d10 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.480+0000 7f9f13fff640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f1806c8f0 0x7f9f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.480+0000 7f9f13fff640 1 -- 172.21.15.50:0/548543730 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f9f2408c580 con 0x7f9f2c15f700 2024-04-02T16:22:49.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.480+0000 7f9f2b7fe640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f1806c8f0 0x7f9f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.481+0000 7f9f2b7fe640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f1806c8f0 0x7f9f1806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c001bb0 tx=0x7f9f1c001b40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.485+0000 7f9f13fff640 1 -- 172.21.15.50:0/548543730 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9f24025020 con 0x7f9f2c15f700 2024-04-02T16:22:49.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.668+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f2c075760 con 0x7f9f2c15f700 2024-04-02T16:22:49.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.668+0000 7f9f13fff640 1 -- 172.21.15.50:0/548543730 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9f2405a590 con 0x7f9f2c15f700 2024-04-02T16:22:49.670 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f1806c8f0 msgr2=0x7f9f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f1806c8f0 0x7f9f1806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c001bb0 tx=0x7f9f1c001b40 comp rx=0 tx=0).stop 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c15f700 msgr2=0x7f9f2c0b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c15f700 0x7f9f2c0b8010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c08e620 tx=0x7f9f24010590 comp rx=0 tx=0).stop 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 shutdown_connections 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9f1806c8f0 0x7f9f1806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c15f700 0x7f9f2c0b8010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c149320 0x7f9f2c15f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 --2- 172.21.15.50:0/548543730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f14003f30 0x7f9f2c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.670+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 >> 172.21.15.50:0/548543730 conn(0x7f9f2c09d5e0 msgr2=0x7f9f2c0d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:49.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.671+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 shutdown_connections 2024-04-02T16:22:49.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.671+0000 7f9f31dba640 1 -- 172.21.15.50:0/548543730 wait complete. 2024-04-02T16:22:49.682 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:49.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.786+0000 7f64b4422640 1 Processor -- start 2024-04-02T16:22:49.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.786+0000 7f64b4422640 1 -- start start 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b4422640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64ac161c80 0x7f64ac0d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b4422640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b4422640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f64ac0d1200 0x7f64ac0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b4422640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0b10e0 con 0x7f64ac161c80 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b4422640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac0b1380 con 0x7f64ac0d1200 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b4422640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac0b1660 con 0x7f64ac0d2e50 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b21be640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64ac161c80 0x7f64ac0d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b21be640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64ac161c80 0x7f64ac0d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38094/0 (socket says 172.21.15.50:38094) 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b21be640 1 -- 172.21.15.50:0/651999438 learned_addr learned my addr 172.21.15.50:0/651999438 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b29bf640 1 --2- 172.21.15.50:0/651999438 >> v2:172.21.15.188:3300/0 conn(0x7f64ac0d1200 0x7f64ac0cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.787+0000 7f64b19bd640 1 --2- 172.21.15.50:0/651999438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b21be640 1 -- 172.21.15.50:0/651999438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 msgr2=0x7f64ac0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b21be640 1 --2- 172.21.15.50:0/651999438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b21be640 1 -- 172.21.15.50:0/651999438 >> v2:172.21.15.188:3300/0 conn(0x7f64ac0d1200 msgr2=0x7f64ac0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b21be640 1 --2- 172.21.15.50:0/651999438 >> v2:172.21.15.188:3300/0 conn(0x7f64ac0d1200 0x7f64ac0cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b21be640 1 -- 172.21.15.50:0/651999438 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0d3340 con 0x7f64ac161c80 2024-04-02T16:22:49.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b21be640 1 --2- 172.21.15.50:0/651999438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac161c80 0x7f64ac0d2910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009870 tx=0x7f64a802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f32c4041efb68de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b11bc640 1 -- 172.21.15.50:0/651999438 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80317c0 con 0x7f64ac161c80 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b11bc640 1 -- 172.21.15.50:0/651999438 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a8037070 con 0x7f64ac161c80 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.788+0000 7f64b11bc640 1 -- 172.21.15.50:0/651999438 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8031b80 con 0x7f64ac161c80 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 -- 172.21.15.50:0/651999438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac161c80 msgr2=0x7f64ac0d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 --2- 172.21.15.50:0/651999438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac161c80 0x7f64ac0d2910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009870 tx=0x7f64a802b860 comp rx=0 tx=0).stop 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 -- 172.21.15.50:0/651999438 shutdown_connections 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 --2- 172.21.15.50:0/651999438 >> v2:172.21.15.188:3300/0 conn(0x7f64ac0d1200 0x7f64ac0cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 --2- 172.21.15.50:0/651999438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 --2- 172.21.15.50:0/651999438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac161c80 0x7f64ac0d2910 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 -- 172.21.15.50:0/651999438 >> 172.21.15.50:0/651999438 conn(0x7f64ac09d5e0 msgr2=0x7f64ac0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 -- 172.21.15.50:0/651999438 shutdown_connections 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.789+0000 7f64b4422640 1 -- 172.21.15.50:0/651999438 wait complete. 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.790+0000 7f64b4422640 1 Processor -- start 2024-04-02T16:22:49.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.790+0000 7f64b4422640 1 -- start start 2024-04-02T16:22:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b4422640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac0d1200 0x7f64ac15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b4422640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b4422640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ac161c80 0x7f64ac0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b4422640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac0cf460 con 0x7f64ac0d1200 2024-04-02T16:22:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b4422640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac0d0390 con 0x7f64ac161c80 2024-04-02T16:22:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b4422640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac0cf5e0 con 0x7f64ac0d2e50 2024-04-02T16:22:49.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac0d1200 0x7f64ac15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac0d1200 0x7f64ac15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38110/0 (socket says 172.21.15.50:38110) 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 -- 172.21.15.50:0/2254588395 learned_addr learned my addr 172.21.15.50:0/2254588395 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b29bf640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ac161c80 0x7f64ac0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b19bd640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 -- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 msgr2=0x7f64ac0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 -- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ac161c80 msgr2=0x7f64ac0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ac161c80 0x7f64ac0cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b21be640 1 -- 172.21.15.50:0/2254588395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0cc650 con 0x7f64ac0d1200 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.791+0000 7f64b29bf640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ac161c80 0x7f64ac0cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.792+0000 7f64b21be640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac0d1200 0x7f64ac15d2b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097c0 tx=0x7f64a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.792+0000 7f64a2ffd640 1 -- 172.21.15.50:0/2254588395 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8032650 con 0x7f64ac0d1200 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.792+0000 7f64a2ffd640 1 -- 172.21.15.50:0/2254588395 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a8032810 con 0x7f64ac0d1200 2024-04-02T16:22:49.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.792+0000 7f64a2ffd640 1 -- 172.21.15.50:0/2254588395 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8032c00 con 0x7f64ac0d1200 2024-04-02T16:22:49.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.792+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac0cc930 con 0x7f64ac0d1200 2024-04-02T16:22:49.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.792+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac0ccef0 con 0x7f64ac0d1200 2024-04-02T16:22:49.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.793+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac075760 con 0x7f64ac0d1200 2024-04-02T16:22:49.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.793+0000 7f64a2ffd640 1 -- 172.21.15.50:0/2254588395 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64a804c060 con 0x7f64ac0d1200 2024-04-02T16:22:49.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.794+0000 7f64a2ffd640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f649006c8f0 0x7f649006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:49.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.794+0000 7f64a2ffd640 1 -- 172.21.15.50:0/2254588395 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f64a80ad660 con 0x7f64ac0d1200 2024-04-02T16:22:49.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.794+0000 7f64b19bd640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f649006c8f0 0x7f649006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:49.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.795+0000 7f64b19bd640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f649006c8f0 0x7f649006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0b7e20 tx=0x7f649c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:49.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.798+0000 7f64a2ffd640 1 -- 172.21.15.50:0/2254588395 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f64a8040070 con 0x7f64ac0d1200 2024-04-02T16:22:49.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.981+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64ac0b9610 con 0x7f64ac0d1200 2024-04-02T16:22:49.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.982+0000 7f64a2ffd640 1 -- 172.21.15.50:0/2254588395 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f64a807b670 con 0x7f64ac0d1200 2024-04-02T16:22:49.983 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f649006c8f0 msgr2=0x7f649006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f649006c8f0 0x7f649006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0b7e20 tx=0x7f649c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac0d1200 msgr2=0x7f64ac15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac0d1200 0x7f64ac15d2b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097c0 tx=0x7f64a80046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 shutdown_connections 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64ac161c80 0x7f64ac0cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64ac0d2e50 0x7f64ac0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f649006c8f0 0x7f649006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 --2- 172.21.15.50:0/2254588395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64ac0d1200 0x7f64ac15d2b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 >> 172.21.15.50:0/2254588395 conn(0x7f64ac09d5e0 msgr2=0x7f64ac0d35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 shutdown_connections 2024-04-02T16:22:49.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:49.984+0000 7f64b4422640 1 -- 172.21.15.50:0/2254588395 wait complete. 2024-04-02T16:22:49.996 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:50.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.100+0000 7fa6f007b640 1 Processor -- start 2024-04-02T16:22:50.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.100+0000 7fa6f007b640 1 -- start start 2024-04-02T16:22:50.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.100+0000 7fa6f007b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e813e120 0x7fa6e80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.100+0000 7fa6f007b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6e8141500 0x7fa6e8140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.100+0000 7fa6f007b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e80cf450 con 0x7fa6e80cef40 2024-04-02T16:22:50.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.100+0000 7fa6f007b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa6e80cf730 con 0x7fa6e8141500 2024-04-02T16:22:50.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6f007b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e80cfa10 con 0x7fa6e813e120 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ede17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e813e120 0x7fa6e80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ed616640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6e8141500 0x7fa6e8140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ee618640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa6e80cef40 0x7fa6e815ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:42796/0 (socket says 172.21.15.50:42796) 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ee618640 1 -- 172.21.15.50:0/2757058183 learned_addr learned my addr 172.21.15.50:0/2757058183 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ede17640 1 -- 172.21.15.50:0/2757058183 >> v2:172.21.15.188:3300/0 conn(0x7fa6e8141500 msgr2=0x7fa6e8140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ede17640 1 --2- 172.21.15.50:0/2757058183 >> v2:172.21.15.188:3300/0 conn(0x7fa6e8141500 0x7fa6e8140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ede17640 1 -- 172.21.15.50:0/2757058183 >> v1:172.21.15.188:6789/0 conn(0x7fa6e80cef40 legacy=0x7fa6e815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ede17640 1 -- 172.21.15.50:0/2757058183 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e80cfcf0 con 0x7fa6e813e120 2024-04-02T16:22:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.101+0000 7fa6ed616640 1 --2- 172.21.15.50:0/2757058183 >> v2:172.21.15.188:3300/0 conn(0x7fa6e8141500 0x7fa6e8140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:50.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.102+0000 7fa6ede17640 1 --2- 172.21.15.50:0/2757058183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e813e120 0x7fa6e80cea00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40097a0 tx=0x7fa6e402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed18da364106caf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.102+0000 7fa6ece15640 1 -- 172.21.15.50:0/2757058183 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4035020 con 0x7fa6e813e120 2024-04-02T16:22:50.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.102+0000 7fa6ece15640 1 -- 172.21.15.50:0/2757058183 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e4031830 con 0x7fa6e813e120 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.102+0000 7fa6ece15640 1 -- 172.21.15.50:0/2757058183 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4031c00 con 0x7fa6e813e120 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 -- 172.21.15.50:0/2757058183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e813e120 msgr2=0x7fa6e80cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 --2- 172.21.15.50:0/2757058183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e813e120 0x7fa6e80cea00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40097a0 tx=0x7fa6e402b8d0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 -- 172.21.15.50:0/2757058183 shutdown_connections 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 --2- 172.21.15.50:0/2757058183 >> v2:172.21.15.188:3300/0 conn(0x7fa6e8141500 0x7fa6e8140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 --2- 172.21.15.50:0/2757058183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e813e120 0x7fa6e80cea00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 -- 172.21.15.50:0/2757058183 >> 172.21.15.50:0/2757058183 conn(0x7fa6e806eb90 msgr2=0x7fa6e806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 -- 172.21.15.50:0/2757058183 shutdown_connections 2024-04-02T16:22:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.103+0000 7fa6f007b640 1 -- 172.21.15.50:0/2757058183 wait complete. 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.104+0000 7fa6f007b640 1 Processor -- start 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.104+0000 7fa6f007b640 1 -- start start 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6f007b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6e80cef40 0x7fa6e8154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6f007b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e813e120 0x7fa6e8136650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6f007b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8141500 0x7fa6e8136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6f007b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e8072930 con 0x7fa6e80cef40 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6f007b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e813e500 con 0x7fa6e813e120 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6f007b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e8072ab0 con 0x7fa6e8141500 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6e80cef40 0x7fa6e8154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6e80cef40 0x7fa6e8154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38124/0 (socket says 172.21.15.50:38124) 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 -- 172.21.15.50:0/3026560203 learned_addr learned my addr 172.21.15.50:0/3026560203 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ee618640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8141500 0x7fa6e8136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ed616640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e813e120 0x7fa6e8136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 -- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8141500 msgr2=0x7fa6e8136b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8141500 0x7fa6e8136b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 -- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e813e120 msgr2=0x7fa6e8136650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e813e120 0x7fa6e8136650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 -- 172.21.15.50:0/3026560203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e81437a0 con 0x7fa6e80cef40 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6ede17640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6e80cef40 0x7fa6e8154a80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40098c0 tx=0x7fa6e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.105+0000 7fa6da7fc640 1 -- 172.21.15.50:0/3026560203 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4041070 con 0x7fa6e80cef40 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.106+0000 7fa6da7fc640 1 -- 172.21.15.50:0/3026560203 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e403f070 con 0x7fa6e80cef40 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.106+0000 7fa6da7fc640 1 -- 172.21.15.50:0/3026560203 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4035040 con 0x7fa6e80cef40 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.106+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e80ad990 con 0x7fa6e80cef40 2024-04-02T16:22:50.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.106+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e8137dd0 con 0x7fa6e80cef40 2024-04-02T16:22:50.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.106+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e8075760 con 0x7fa6e80cef40 2024-04-02T16:22:50.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.107+0000 7fa6da7fc640 1 -- 172.21.15.50:0/3026560203 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e4038020 con 0x7fa6e80cef40 2024-04-02T16:22:50.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.107+0000 7fa6da7fc640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.107+0000 7fa6da7fc640 1 -- 172.21.15.50:0/3026560203 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fa6e40ad880 con 0x7fa6e80cef40 2024-04-02T16:22:50.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.107+0000 7fa6ed616640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.108+0000 7fa6ed616640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009560 tx=0x7fa6dc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.112+0000 7fa6da7fc640 1 -- 172.21.15.50:0/3026560203 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa6e4031800 con 0x7fa6e80cef40 2024-04-02T16:22:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.295+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6e80734f0 con 0x7fa6e80cef40 2024-04-02T16:22:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.295+0000 7fa6da7fc640 1 -- 172.21.15.50:0/3026560203 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa6e407b920 con 0x7fa6e80cef40 2024-04-02T16:22:50.296 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:50.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6cc06c8f0 msgr2=0x7fa6cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009560 tx=0x7fa6dc008040 comp rx=0 tx=0).stop 2024-04-02T16:22:50.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6e80cef40 msgr2=0x7fa6e8154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6e80cef40 0x7fa6e8154a80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40098c0 tx=0x7fa6e40046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 shutdown_connections 2024-04-02T16:22:50.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6e8141500 0x7fa6e8136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e813e120 0x7fa6e8136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 --2- 172.21.15.50:0/3026560203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6e80cef40 0x7fa6e8154a80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 >> 172.21.15.50:0/3026560203 conn(0x7fa6e806eb90 msgr2=0x7fa6e8140a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:50.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 shutdown_connections 2024-04-02T16:22:50.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.297+0000 7fa6f007b640 1 -- 172.21.15.50:0/3026560203 wait complete. 2024-04-02T16:22:50.310 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:50.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.414+0000 7ff39b0c7640 1 Processor -- start 2024-04-02T16:22:50.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.414+0000 7ff39b0c7640 1 -- start start 2024-04-02T16:22:50.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.414+0000 7ff39b0c7640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff3940d4980 0x7ff394162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.414+0000 7ff39b0c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940d0c80 0x7ff3940d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.414+0000 7ff39b0c7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3940d5000 con 0x7ff394161ca0 2024-04-02T16:22:50.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.414+0000 7ff39b0c7640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ff3941630f0 con 0x7ff3940d4980 2024-04-02T16:22:50.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.414+0000 7ff39b0c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940d15f0 con 0x7ff3940d0c80 2024-04-02T16:22:50.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff38bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff3940d4980 0x7ff394162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff38bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff3940d4980 0x7ff394162bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38130/0 (socket says 172.21.15.50:38130) 2024-04-02T16:22:50.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff38bfff640 1 -- 172.21.15.50:0/1208180142 learned_addr learned my addr 172.21.15.50:0/1208180142 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:50.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff399664640 1 --2- 172.21.15.50:0/1208180142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940d0c80 0x7ff3940d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff38bfff640 1 -- 172.21.15.50:0/1208180142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940d0c80 msgr2=0x7ff3940d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:50.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff38bfff640 1 --2- 172.21.15.50:0/1208180142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940d0c80 0x7ff3940d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff38bfff640 1 -- 172.21.15.50:0/1208180142 >> v1:172.21.15.188:6789/0 conn(0x7ff394161ca0 legacy=0x7ff3940cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.415+0000 7ff38bfff640 1 -- 172.21.15.50:0/1208180142 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940cf0d0 con 0x7ff3940d4980 2024-04-02T16:22:50.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff38bfff640 1 --2- 172.21.15.50:0/1208180142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff394162bb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff3840098a0 tx=0x7ff38402b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7528ccc1bf1327 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff38b7fe640 1 -- 172.21.15.50:0/1208180142 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384038070 con 0x7ff3940d4980 2024-04-02T16:22:50.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff38b7fe640 1 -- 172.21.15.50:0/1208180142 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384031520 con 0x7ff3940d4980 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff38b7fe640 1 -- 172.21.15.50:0/1208180142 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840318f0 con 0x7ff3940d4980 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 -- 172.21.15.50:0/1208180142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 msgr2=0x7ff394162bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/1208180142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff394162bb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff3840098a0 tx=0x7ff38402b7c0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 -- 172.21.15.50:0/1208180142 shutdown_connections 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/1208180142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff394162bb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/1208180142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940d0c80 0x7ff3940d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 -- 172.21.15.50:0/1208180142 >> 172.21.15.50:0/1208180142 conn(0x7ff39409d5c0 msgr2=0x7ff3940d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 -- 172.21.15.50:0/1208180142 shutdown_connections 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.416+0000 7ff39b0c7640 1 -- 172.21.15.50:0/1208180142 wait complete. 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.417+0000 7ff39b0c7640 1 Processor -- start 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.417+0000 7ff39b0c7640 1 -- start start 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff39b0c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3940d0c80 0x7ff39415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff39b0c7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff3941e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff39b0c7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff394161ca0 0x7ff3940c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff39b0c7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940ced80 con 0x7ff3940d4980 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff39b0c7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940cf690 con 0x7ff3940d0c80 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff39b0c7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940b70c0 con 0x7ff394161ca0 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff38bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff3941e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff38bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff3941e8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38134/0 (socket says 172.21.15.50:38134) 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff38bfff640 1 -- 172.21.15.50:0/2049430306 learned_addr learned my addr 172.21.15.50:0/2049430306 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.418+0000 7ff398e63640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3940d0c80 0x7ff39415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff399664640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff394161ca0 0x7ff3940c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff38bfff640 1 -- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff394161ca0 msgr2=0x7ff3940c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff38bfff640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff394161ca0 0x7ff3940c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff38bfff640 1 -- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3940d0c80 msgr2=0x7ff39415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff38bfff640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3940d0c80 0x7ff39415cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff38bfff640 1 -- 172.21.15.50:0/2049430306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3941e9660 con 0x7ff3940d4980 2024-04-02T16:22:50.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff38bfff640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff3941e8cf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff384000c00 tx=0x7ff3840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff3897fa640 1 -- 172.21.15.50:0/2049430306 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384042070 con 0x7ff3940d4980 2024-04-02T16:22:50.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff3897fa640 1 -- 172.21.15.50:0/2049430306 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840338a0 con 0x7ff3940d4980 2024-04-02T16:22:50.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3941e9940 con 0x7ff3940d4980 2024-04-02T16:22:50.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff3897fa640 1 -- 172.21.15.50:0/2049430306 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384038040 con 0x7ff3940d4980 2024-04-02T16:22:50.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.419+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3940c8ca0 con 0x7ff3940d4980 2024-04-02T16:22:50.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.420+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff394075760 con 0x7ff3940d4980 2024-04-02T16:22:50.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.421+0000 7ff3897fa640 1 -- 172.21.15.50:0/2049430306 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff384045070 con 0x7ff3940d4980 2024-04-02T16:22:50.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.421+0000 7ff3897fa640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff38006c8f0 0x7ff38006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.421+0000 7ff3897fa640 1 -- 172.21.15.50:0/2049430306 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff3840ada30 con 0x7ff3940d4980 2024-04-02T16:22:50.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.421+0000 7ff398e63640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff38006c8f0 0x7ff38006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.422+0000 7ff398e63640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff38006c8f0 0x7ff38006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff390000f60 tx=0x7ff390009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.425+0000 7ff3897fa640 1 -- 172.21.15.50:0/2049430306 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff38407b510 con 0x7ff3940d4980 2024-04-02T16:22:50.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.608+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff3940ba130 con 0x7ff3940d4980 2024-04-02T16:22:50.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.609+0000 7ff3897fa640 1 -- 172.21.15.50:0/2049430306 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff38407f810 con 0x7ff3940d4980 2024-04-02T16:22:50.610 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:50.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff38006c8f0 msgr2=0x7ff38006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff38006c8f0 0x7ff38006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff390000f60 tx=0x7ff390009040 comp rx=0 tx=0).stop 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 msgr2=0x7ff3941e8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff3941e8cf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff384000c00 tx=0x7ff3840046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 shutdown_connections 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff394161ca0 0x7ff3940c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff38006c8f0 0x7ff38006edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3940d4980 0x7ff3941e8cf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 --2- 172.21.15.50:0/2049430306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3940d0c80 0x7ff39415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 >> 172.21.15.50:0/2049430306 conn(0x7ff39409d5c0 msgr2=0x7ff3940d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 shutdown_connections 2024-04-02T16:22:50.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.611+0000 7ff39b0c7640 1 -- 172.21.15.50:0/2049430306 wait complete. 2024-04-02T16:22:50.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-04-02T16:22:50.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.724+0000 7f89685a3640 1 Processor -- start 2024-04-02T16:22:50.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.724+0000 7f89685a3640 1 -- start start 2024-04-02T16:22:50.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f89685a3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89600b4c20 0x7f8960142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f89685a3640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89600b5400 con 0x7f89600b1bc0 2024-04-02T16:22:50.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f89685a3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89600b5120 con 0x7f89600b3b40 2024-04-02T16:22:50.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f89685a3640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f89600b3060 con 0x7f89600b4c20 2024-04-02T16:22:50.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f8966b40640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89600b4c20 0x7f8960142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f896633f640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f89600b1bc0 0x7f89600b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34646/0 (socket says 172.21.15.50:34646) 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f8966b40640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f89600b4c20 0x7f8960142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38138/0 (socket says 172.21.15.50:38138) 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f896633f640 1 -- 172.21.15.50:0/3664581855 learned_addr learned my addr 172.21.15.50:0/3664581855 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f8966b40640 1 -- 172.21.15.50:0/3664581855 >> v1:172.21.15.188:6789/0 conn(0x7f89600b3b40 legacy=0x7f89600b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f8966b40640 1 -- 172.21.15.50:0/3664581855 >> v1:172.21.15.50:6789/0 conn(0x7f89600b1bc0 legacy=0x7f89600b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f896533d640 1 -- 172.21.15.50:0/3664581855 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099111958 0 0) 0x7f89600b5400 con 0x7f89600b1bc0 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.725+0000 7f8966b40640 1 -- 172.21.15.50:0/3664581855 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89600b3340 con 0x7f89600b4c20 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.726+0000 7f8966b40640 1 --2- 172.21.15.50:0/3664581855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f8960142fb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f895c009d90 tx=0x7f895c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1d4795c81b3f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.726+0000 7f896533d640 1 -- 172.21.15.50:0/3664581855 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c039070 con 0x7f89600b4c20 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.726+0000 7f896533d640 1 -- 172.21.15.50:0/3664581855 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c02cc60 con 0x7f89600b4c20 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.726+0000 7f896533d640 1 -- 172.21.15.50:0/3664581855 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c034660 con 0x7f89600b4c20 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.727+0000 7f89685a3640 1 -- 172.21.15.50:0/3664581855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 msgr2=0x7f8960142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.727+0000 7f89685a3640 1 --2- 172.21.15.50:0/3664581855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f8960142fb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f895c009d90 tx=0x7f895c02beb0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.727+0000 7f89685a3640 1 -- 172.21.15.50:0/3664581855 shutdown_connections 2024-04-02T16:22:50.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.727+0000 7f89685a3640 1 --2- 172.21.15.50:0/3664581855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f8960142fb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.727+0000 7f89685a3640 1 -- 172.21.15.50:0/3664581855 >> 172.21.15.50:0/3664581855 conn(0x7f896006ebf0 msgr2=0x7f896006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:50.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.727+0000 7f89685a3640 1 -- 172.21.15.50:0/3664581855 shutdown_connections 2024-04-02T16:22:50.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.727+0000 7f89685a3640 1 -- 172.21.15.50:0/3664581855 wait complete. 2024-04-02T16:22:50.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.728+0000 7f89685a3640 1 Processor -- start 2024-04-02T16:22:50.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.728+0000 7f89685a3640 1 -- start start 2024-04-02T16:22:50.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f89685a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89600b1bc0 0x7f8960138c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f89685a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89600b3b40 0x7f896013b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f89685a3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f896015e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f89685a3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8960142c80 con 0x7f89600b4c20 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f89685a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8960142b00 con 0x7f89600b1bc0 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f89685a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f89601434f0 con 0x7f89600b3b40 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f896015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f896015e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38144/0 (socket says 172.21.15.50:38144) 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 -- 172.21.15.50:0/2079001006 learned_addr learned my addr 172.21.15.50:0/2079001006 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f896633f640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89600b1bc0 0x7f8960138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8965b3e640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89600b3b40 0x7f896013b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 -- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89600b3b40 msgr2=0x7f896013b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89600b3b40 0x7f896013b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 -- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89600b1bc0 msgr2=0x7f8960138c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89600b1bc0 0x7f8960138c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:50.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 -- 172.21.15.50:0/2079001006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896013a470 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f896633f640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89600b1bc0 0x7f8960138c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8965b3e640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89600b3b40 0x7f896013b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8966b40640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f896015e650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f895c037040 tx=0x7f895c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8956ffd640 1 -- 172.21.15.50:0/2079001006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c044020 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.729+0000 7f8956ffd640 1 -- 172.21.15.50:0/2079001006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c03a040 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.730+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896013a750 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.730+0000 7f8956ffd640 1 -- 172.21.15.50:0/2079001006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c039050 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.730+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896013ad10 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.731+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89600756f0 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.731+0000 7f8956ffd640 1 -- 172.21.15.50:0/2079001006 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f895c02cb20 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.731+0000 7f8956ffd640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f894406c8f0 0x7f894406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.731+0000 7f8956ffd640 1 -- 172.21.15.50:0/2079001006 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f895c0ae6b0 con 0x7f89600b4c20 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.731+0000 7f896633f640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f894406c8f0 0x7f894406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:50.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.732+0000 7f896633f640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f894406c8f0 0x7f894406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8958000f60 tx=0x7f8958009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:50.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.736+0000 7f8956ffd640 1 -- 172.21.15.50:0/2079001006 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f895c04e060 con 0x7f89600b4c20 2024-04-02T16:22:50.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:50.992+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7f896006c280 con 0x7f89600b4c20 2024-04-02T16:22:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.773+0000 7f8956ffd640 1 -- 172.21.15.50:0/2079001006 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v71) ==== 147+0+0 (secure 0 0 0) 0x7f895c07c750 con 0x7f89600b4c20 2024-04-02T16:22:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f894406c8f0 msgr2=0x7f894406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f894406c8f0 0x7f894406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8958000f60 tx=0x7f8958009040 comp rx=0 tx=0).stop 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 msgr2=0x7f896015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f896015e650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f895c037040 tx=0x7f895c004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 shutdown_connections 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f894406c8f0 0x7f894406edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89600b4c20 0x7f896015e650 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89600b3b40 0x7f896013b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 --2- 172.21.15.50:0/2079001006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89600b1bc0 0x7f8960138c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 >> 172.21.15.50:0/2079001006 conn(0x7f896006ebf0 msgr2=0x7f89601605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 shutdown_connections 2024-04-02T16:22:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.776+0000 7f89685a3640 1 -- 172.21.15.50:0/2079001006 wait complete. 2024-04-02T16:22:51.791 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-02T16:22:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.901+0000 7f5f2c76d640 1 Processor -- start 2024-04-02T16:22:51.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.901+0000 7f5f2c76d640 1 -- start start 2024-04-02T16:22:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2c76d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5f24145c60 0x7f5f2406b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2c76d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f240ce1a0 0x7f5f2406ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2c76d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f240cfdc0 con 0x7f5f240d0b80 2024-04-02T16:22:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2c76d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f5f24146040 con 0x7f5f24145c60 2024-04-02T16:22:51.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2c76d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5f240d00a0 con 0x7f5f240ce1a0 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2a509640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5f24145c60 0x7f5f2406b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f29d08640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f5f240d0b80 0x7f5f240cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34662/0 (socket says 172.21.15.50:34662) 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2a509640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f5f24145c60 0x7f5f2406b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38156/0 (socket says 172.21.15.50:38156) 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f29d08640 1 -- 172.21.15.50:0/3967633309 learned_addr learned my addr 172.21.15.50:0/3967633309 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2ad0a640 1 --2- 172.21.15.50:0/3967633309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f240ce1a0 0x7f5f2406ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2a509640 1 -- 172.21.15.50:0/3967633309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f240ce1a0 msgr2=0x7f5f2406ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2a509640 1 --2- 172.21.15.50:0/3967633309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f240ce1a0 0x7f5f2406ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2a509640 1 -- 172.21.15.50:0/3967633309 >> v1:172.21.15.50:6789/0 conn(0x7f5f240d0b80 legacy=0x7f5f240cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f29507640 1 -- 172.21.15.50:0/3967633309 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2711716896 0 0) 0x7f5f240cfdc0 con 0x7f5f240d0b80 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.902+0000 7f5f2a509640 1 -- 172.21.15.50:0/3967633309 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f240d0380 con 0x7f5f24145c60 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.903+0000 7f5f2a509640 1 --2- 172.21.15.50:0/3967633309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f24145c60 0x7f5f2406b520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0097a0 tx=0x7f5f1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e71371ebc04f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.903+0000 7f5f29507640 1 -- 172.21.15.50:0/3967633309 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0315a0 con 0x7f5f24145c60 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.903+0000 7f5f29507640 1 -- 172.21.15.50:0/3967633309 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c037070 con 0x7f5f24145c60 2024-04-02T16:22:51.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.903+0000 7f5f29507640 1 -- 172.21.15.50:0/3967633309 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c031960 con 0x7f5f24145c60 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3967633309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f24145c60 msgr2=0x7f5f2406b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3967633309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f24145c60 0x7f5f2406b520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0097a0 tx=0x7f5f1c02b840 comp rx=0 tx=0).stop 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3967633309 shutdown_connections 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3967633309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f240ce1a0 0x7f5f2406ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3967633309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f24145c60 0x7f5f2406b520 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3967633309 >> 172.21.15.50:0/3967633309 conn(0x7f5f240b2db0 msgr2=0x7f5f240b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3967633309 shutdown_connections 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.904+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3967633309 wait complete. 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.905+0000 7f5f2c76d640 1 Processor -- start 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.905+0000 7f5f2c76d640 1 -- start start 2024-04-02T16:22:51.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2c76d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f240ce1a0 0x7f5f2415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2c76d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f240d0b80 0x7f5f240cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2c76d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f24145c60 0x7f5f240a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2c76d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2415c890 con 0x7f5f240ce1a0 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2c76d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5f240cfad0 con 0x7f5f240d0b80 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2c76d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2415ca10 con 0x7f5f24145c60 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f240ce1a0 0x7f5f2415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f240ce1a0 0x7f5f2415bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38170/0 (socket says 172.21.15.50:38170) 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 -- 172.21.15.50:0/3289676221 learned_addr learned my addr 172.21.15.50:0/3289676221 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 -- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f24145c60 msgr2=0x7f5f240a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f24145c60 0x7f5f240a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 -- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f240d0b80 msgr2=0x7f5f240cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f240d0b80 0x7f5f240cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 -- 172.21.15.50:0/3289676221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f240cb730 con 0x7f5f240ce1a0 2024-04-02T16:22:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.906+0000 7f5f2a509640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f240ce1a0 0x7f5f2415bca0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c034040 tx=0x7f5f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.907+0000 7f5f1affd640 1 -- 172.21.15.50:0/3289676221 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c032630 con 0x7f5f240ce1a0 2024-04-02T16:22:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.907+0000 7f5f1affd640 1 -- 172.21.15.50:0/3289676221 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c0327f0 con 0x7f5f240ce1a0 2024-04-02T16:22:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.907+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f240cb9d0 con 0x7f5f240ce1a0 2024-04-02T16:22:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.907+0000 7f5f1affd640 1 -- 172.21.15.50:0/3289676221 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c032be0 con 0x7f5f240ce1a0 2024-04-02T16:22:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.907+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f240cbf90 con 0x7f5f240ce1a0 2024-04-02T16:22:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.908+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f24075760 con 0x7f5f240ce1a0 2024-04-02T16:22:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.908+0000 7f5f1affd640 1 -- 172.21.15.50:0/3289676221 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f1c03b3f0 con 0x7f5f240ce1a0 2024-04-02T16:22:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.908+0000 7f5f1affd640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5f0806c8f0 0x7f5f0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.909+0000 7f5f1affd640 1 -- 172.21.15.50:0/3289676221 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f5f1c02f030 con 0x7f5f240ce1a0 2024-04-02T16:22:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.909+0000 7f5f29d08640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5f0806c8f0 0x7f5f0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.909+0000 7f5f29d08640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5f0806c8f0 0x7f5f0806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5f14007fd0 tx=0x7f5f14007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:51.913+0000 7f5f1affd640 1 -- 172.21.15.50:0/3289676221 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5f1c0b4080 con 0x7f5f240ce1a0 2024-04-02T16:22:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.095+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5f240cedd0 con 0x7f5f240ce1a0 2024-04-02T16:22:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.099+0000 7f5f1affd640 1 -- 172.21.15.50:0/3289676221 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f5f1c040070 con 0x7f5f240ce1a0 2024-04-02T16:22:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5f0806c8f0 msgr2=0x7f5f0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5f0806c8f0 0x7f5f0806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5f14007fd0 tx=0x7f5f14007920 comp rx=0 tx=0).stop 2024-04-02T16:22:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f240ce1a0 msgr2=0x7f5f2415bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f240ce1a0 0x7f5f2415bca0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c034040 tx=0x7f5f1c0046c0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 shutdown_connections 2024-04-02T16:22:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f24145c60 0x7f5f240a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f240d0b80 0x7f5f240cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f5f0806c8f0 0x7f5f0806edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 --2- 172.21.15.50:0/3289676221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f240ce1a0 0x7f5f2415bca0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 >> 172.21.15.50:0/3289676221 conn(0x7f5f240b2db0 msgr2=0x7f5f2406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 shutdown_connections 2024-04-02T16:22:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.101+0000 7f5f2c76d640 1 -- 172.21.15.50:0/3289676221 wait complete. 2024-04-02T16:22:52.113 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-02T16:22:52.113 DEBUG:teuthology.orchestra.run.smithi050:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-02T16:22:52.151 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-02T16:22:52.205 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-02T16:22:52.206 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:52.206 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:52.220 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:52.221 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:52.276 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:52.276 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:52.331 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:52.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.471+0000 7faca933d640 1 Processor -- start 2024-04-02T16:22:52.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.471+0000 7faca933d640 1 -- start start 2024-04-02T16:22:52.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca933d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406e8b0 0x7faca406ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca933d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faca40d0a80 0x7faca4072320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca933d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca406f1d0 con 0x7faca40d0620 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca933d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7faca415d3d0 con 0x7faca40d0a80 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca933d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faca415d6b0 con 0x7faca406e8b0 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca27fc640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7faca40d0620 0x7faca415c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34664/0 (socket says 172.21.15.50:34664) 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7faca40d0a80 0x7faca4072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.472+0000 7faca27fc640 1 -- 172.21.15.50:0/1314861773 learned_addr learned my addr 172.21.15.50:0/1314861773 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.473+0000 7faca2ffd640 1 --2- 172.21.15.50:0/1314861773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406e8b0 0x7faca406ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.473+0000 7faca37fe640 1 -- 172.21.15.50:0/1314861773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406e8b0 msgr2=0x7faca406ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.473+0000 7faca37fe640 1 --2- 172.21.15.50:0/1314861773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406e8b0 0x7faca406ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.473+0000 7faca37fe640 1 -- 172.21.15.50:0/1314861773 >> v1:172.21.15.50:6789/0 conn(0x7faca40d0620 legacy=0x7faca415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.473+0000 7faca37fe640 1 -- 172.21.15.50:0/1314861773 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca40cfe20 con 0x7faca40d0a80 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.473+0000 7faca37fe640 1 --2- 172.21.15.50:0/1314861773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0a80 0x7faca4072320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20d5856ac138683 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca1ffb640 1 -- 172.21.15.50:0/1314861773 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c039070 con 0x7faca40d0a80 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca1ffb640 1 -- 172.21.15.50:0/1314861773 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9c02cc50 con 0x7faca40d0a80 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca1ffb640 1 -- 172.21.15.50:0/1314861773 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c034440 con 0x7faca40d0a80 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 -- 172.21.15.50:0/1314861773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0a80 msgr2=0x7faca4072320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 --2- 172.21.15.50:0/1314861773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0a80 0x7faca4072320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c02bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 -- 172.21.15.50:0/1314861773 shutdown_connections 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 --2- 172.21.15.50:0/1314861773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0a80 0x7faca4072320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 --2- 172.21.15.50:0/1314861773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406e8b0 0x7faca406ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 -- 172.21.15.50:0/1314861773 >> 172.21.15.50:0/1314861773 conn(0x7faca40b6540 msgr2=0x7faca40b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 -- 172.21.15.50:0/1314861773 shutdown_connections 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.474+0000 7faca933d640 1 -- 172.21.15.50:0/1314861773 wait complete. 2024-04-02T16:22:52.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 Processor -- start 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 -- start start 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca406e8b0 0x7faca412caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0620 0x7faca41f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40d0a80 0x7faca41f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faca4145f80 con 0x7faca40d0620 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faca4145e00 con 0x7faca406e8b0 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca933d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faca40cfa40 con 0x7faca40d0a80 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca27fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0620 0x7faca41f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca27fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0620 0x7faca41f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38182/0 (socket says 172.21.15.50:38182) 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.476+0000 7faca27fc640 1 -- 172.21.15.50:0/2305419098 learned_addr learned my addr 172.21.15.50:0/2305419098 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca2ffd640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca406e8b0 0x7faca412caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca37fe640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40d0a80 0x7faca41f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca27fc640 1 -- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40d0a80 msgr2=0x7faca41f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca27fc640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40d0a80 0x7faca41f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca27fc640 1 -- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca406e8b0 msgr2=0x7faca412caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca27fc640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca406e8b0 0x7faca412caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca27fc640 1 -- 172.21.15.50:0/2305419098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca41f57c0 con 0x7faca40d0620 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca37fe640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40d0a80 0x7faca41f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca27fc640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0620 0x7faca41f25b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c400 tx=0x7fac9800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7fac8b7fe640 1 -- 172.21.15.50:0/2305419098 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98017070 con 0x7faca40d0620 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7fac8b7fe640 1 -- 172.21.15.50:0/2305419098 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98005ce0 con 0x7faca40d0620 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca41f5aa0 con 0x7faca40d0620 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7fac8b7fe640 1 -- 172.21.15.50:0/2305419098 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac980072f0 con 0x7faca40d0620 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.477+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca41f6060 con 0x7faca40d0620 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.478+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca40756f0 con 0x7faca40d0620 2024-04-02T16:22:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.478+0000 7fac8b7fe640 1 -- 172.21.15.50:0/2305419098 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac9801a070 con 0x7faca40d0620 2024-04-02T16:22:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.479+0000 7fac8b7fe640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fac9006c8f0 0x7fac9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.479+0000 7fac8b7fe640 1 -- 172.21.15.50:0/2305419098 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fac9808b2f0 con 0x7faca40d0620 2024-04-02T16:22:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.479+0000 7faca2ffd640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fac9006c8f0 0x7fac9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.480+0000 7faca2ffd640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fac9006c8f0 0x7fac9006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac94008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.483+0000 7fac8b7fe640 1 -- 172.21.15.50:0/2305419098 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fac98059390 con 0x7faca40d0620 2024-04-02T16:22:52.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.668+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faca41f6230 con 0x7faca40d0620 2024-04-02T16:22:52.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.669+0000 7fac8b7fe640 1 -- 172.21.15.50:0/2305419098 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7fac98058c90 con 0x7faca40d0620 2024-04-02T16:22:52.670 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:52.670 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fac9006c8f0 msgr2=0x7fac9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fac9006c8f0 0x7fac9006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac94008040 comp rx=0 tx=0).stop 2024-04-02T16:22:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0620 msgr2=0x7faca41f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0620 0x7faca41f25b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c400 tx=0x7fac9800c8c0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 shutdown_connections 2024-04-02T16:22:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40d0a80 0x7faca41f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fac9006c8f0 0x7fac9006edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faca40d0620 0x7faca41f25b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 --2- 172.21.15.50:0/2305419098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca406e8b0 0x7faca412caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 >> 172.21.15.50:0/2305419098 conn(0x7faca40b6540 msgr2=0x7faca40d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 shutdown_connections 2024-04-02T16:22:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.671+0000 7faca933d640 1 -- 172.21.15.50:0/2305419098 wait complete. 2024-04-02T16:22:52.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:52.684 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f972991c640 1 Processor -- start 2024-04-02T16:22:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f972991c640 1 -- start start 2024-04-02T16:22:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f972991c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9724161ca0 0x7f97240d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f972991c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9724162bb0 0x7f97240cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f972991c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97240d5000 con 0x7f97240d0c80 2024-04-02T16:22:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f972991c640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f97240cf0d0 con 0x7f9724161ca0 2024-04-02T16:22:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f972991c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97240cf3b0 con 0x7f9724162bb0 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.788+0000 7f9722ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9724161ca0 0x7f97240d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.789+0000 7f9722ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9724161ca0 0x7f97240d4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38196/0 (socket says 172.21.15.50:38196) 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.789+0000 7f9722ffd640 1 -- 172.21.15.50:0/2154338426 learned_addr learned my addr 172.21.15.50:0/2154338426 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.789+0000 7f97227fc640 1 --2- 172.21.15.50:0/2154338426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9724162bb0 0x7f97240cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.789+0000 7f9722ffd640 1 -- 172.21.15.50:0/2154338426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9724162bb0 msgr2=0x7f97240cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.789+0000 7f9722ffd640 1 --2- 172.21.15.50:0/2154338426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9724162bb0 0x7f97240cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.789+0000 7f9722ffd640 1 -- 172.21.15.50:0/2154338426 >> v1:172.21.15.188:6789/0 conn(0x7f97240d0c80 legacy=0x7f97240b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.789+0000 7f9722ffd640 1 -- 172.21.15.50:0/2154338426 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97240cf690 con 0x7f9724161ca0 2024-04-02T16:22:52.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f9722ffd640 1 --2- 172.21.15.50:0/2154338426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724161ca0 0x7f97240d4980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f971c009870 tx=0x7f971c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52d19c0133a0c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f9721ffb640 1 -- 172.21.15.50:0/2154338426 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c0317c0 con 0x7f9724161ca0 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f9721ffb640 1 -- 172.21.15.50:0/2154338426 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971c037070 con 0x7f9724161ca0 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f9721ffb640 1 -- 172.21.15.50:0/2154338426 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c031b80 con 0x7f9724161ca0 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 -- 172.21.15.50:0/2154338426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724161ca0 msgr2=0x7f97240d4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 --2- 172.21.15.50:0/2154338426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724161ca0 0x7f97240d4980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f971c009870 tx=0x7f971c02b860 comp rx=0 tx=0).stop 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 -- 172.21.15.50:0/2154338426 shutdown_connections 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 --2- 172.21.15.50:0/2154338426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9724162bb0 0x7f97240cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 --2- 172.21.15.50:0/2154338426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724161ca0 0x7f97240d4980 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 -- 172.21.15.50:0/2154338426 >> 172.21.15.50:0/2154338426 conn(0x7f972409d5c0 msgr2=0x7f97240d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 -- 172.21.15.50:0/2154338426 shutdown_connections 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.790+0000 7f972991c640 1 -- 172.21.15.50:0/2154338426 wait complete. 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.791+0000 7f972991c640 1 Processor -- start 2024-04-02T16:22:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f972991c640 1 -- start start 2024-04-02T16:22:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f972991c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d0c80 0x7f97240cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f972991c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724161ca0 0x7f97241f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f972991c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724162bb0 0x7f97240b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f972991c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97240ced80 con 0x7f9724162bb0 2024-04-02T16:22:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f972991c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97240d00f0 con 0x7f9724161ca0 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f972991c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97240b77d0 con 0x7f97240d0c80 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.792+0000 7f97237fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724162bb0 0x7f97240b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724162bb0 0x7f97240b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38212/0 (socket says 172.21.15.50:38212) 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 -- 172.21.15.50:0/1592618709 learned_addr learned my addr 172.21.15.50:0/1592618709 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f9722ffd640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d0c80 0x7f97240cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97227fc640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724161ca0 0x7f97241f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 -- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d0c80 msgr2=0x7f97240cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d0c80 0x7f97240cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 -- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724161ca0 msgr2=0x7f97241f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724161ca0 0x7f97241f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 -- 172.21.15.50:0/1592618709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97241f4c40 con 0x7f9724162bb0 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97227fc640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724161ca0 0x7f97241f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f97237fe640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724162bb0 0x7f97240b8b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f971400c780 tx=0x7f971400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f971bfff640 1 -- 172.21.15.50:0/1592618709 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714004b60 con 0x7f9724162bb0 2024-04-02T16:22:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f971bfff640 1 -- 172.21.15.50:0/1592618709 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9714004d00 con 0x7f9724162bb0 2024-04-02T16:22:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97241f4f20 con 0x7f9724162bb0 2024-04-02T16:22:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f971bfff640 1 -- 172.21.15.50:0/1592618709 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971400aa90 con 0x7f9724162bb0 2024-04-02T16:22:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.793+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97241f54e0 con 0x7f9724162bb0 2024-04-02T16:22:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.794+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9724075760 con 0x7f9724162bb0 2024-04-02T16:22:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.794+0000 7f971bfff640 1 -- 172.21.15.50:0/1592618709 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f971400ac30 con 0x7f9724162bb0 2024-04-02T16:22:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.795+0000 7f971bfff640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f970406c8f0 0x7f970406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.795+0000 7f971bfff640 1 -- 172.21.15.50:0/1592618709 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f971408c4f0 con 0x7f9724162bb0 2024-04-02T16:22:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.795+0000 7f9722ffd640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f970406c8f0 0x7f970406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.796+0000 7f9722ffd640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f970406c8f0 0x7f970406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f971c036000 tx=0x7f971c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.800+0000 7f971bfff640 1 -- 172.21.15.50:0/1592618709 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f971405a590 con 0x7f9724162bb0 2024-04-02T16:22:52.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.982+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97240ba130 con 0x7f9724162bb0 2024-04-02T16:22:52.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.982+0000 7f971bfff640 1 -- 172.21.15.50:0/1592618709 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9714059e90 con 0x7f9724162bb0 2024-04-02T16:22:52.984 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:52.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f970406c8f0 msgr2=0x7f970406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f970406c8f0 0x7f970406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f971c036000 tx=0x7f971c03d040 comp rx=0 tx=0).stop 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724162bb0 msgr2=0x7f97240b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724162bb0 0x7f97240b8b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f971400c780 tx=0x7f971400cc40 comp rx=0 tx=0).stop 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 shutdown_connections 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f970406c8f0 0x7f970406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9724162bb0 0x7f97240b8b80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724161ca0 0x7f97241f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 --2- 172.21.15.50:0/1592618709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240d0c80 0x7f97240cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 >> 172.21.15.50:0/1592618709 conn(0x7f972409d5c0 msgr2=0x7f972415dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 shutdown_connections 2024-04-02T16:22:52.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:52.984+0000 7f972991c640 1 -- 172.21.15.50:0/1592618709 wait complete. 2024-04-02T16:22:52.996 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:53.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.099+0000 7f95ea38d640 1 Processor -- start 2024-04-02T16:22:53.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.099+0000 7f95ea38d640 1 -- start start 2024-04-02T16:22:53.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95ea38d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e4161c80 0x7f95e40d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95ea38d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f95e40d2e50 0x7f95e40d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95ea38d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e40b10e0 con 0x7f95e40d1200 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95ea38d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f95e40b1380 con 0x7f95e40d2e50 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95ea38d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95e40b1660 con 0x7f95e4161c80 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95e37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f95e40d2e50 0x7f95e40d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95e892a640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f95e40d1200 0x7f95e40cef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34680/0 (socket says 172.21.15.50:34680) 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95e892a640 1 -- 172.21.15.50:0/3205665785 learned_addr learned my addr 172.21.15.50:0/3205665785 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.100+0000 7f95e3fff640 1 --2- 172.21.15.50:0/3205665785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e4161c80 0x7f95e40d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.101+0000 7f95e37fe640 1 -- 172.21.15.50:0/3205665785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e4161c80 msgr2=0x7f95e40d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.101+0000 7f95e37fe640 1 --2- 172.21.15.50:0/3205665785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e4161c80 0x7f95e40d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.101+0000 7f95e37fe640 1 -- 172.21.15.50:0/3205665785 >> v1:172.21.15.50:6789/0 conn(0x7f95e40d1200 legacy=0x7f95e40cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.101+0000 7f95e2ffd640 1 -- 172.21.15.50:0/3205665785 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916306848 0 0) 0x7f95e40b10e0 con 0x7f95e40d1200 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.101+0000 7f95e37fe640 1 -- 172.21.15.50:0/3205665785 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e40d3340 con 0x7f95e40d2e50 2024-04-02T16:22:53.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.101+0000 7f95e37fe640 1 --2- 172.21.15.50:0/3205665785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e40d2e50 0x7f95e40d0cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097d0 tx=0x7f95d802b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9d764e2d102850 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95e2ffd640 1 -- 172.21.15.50:0/3205665785 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8031770 con 0x7f95e40d2e50 2024-04-02T16:22:53.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95e2ffd640 1 -- 172.21.15.50:0/3205665785 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d8037070 con 0x7f95e40d2e50 2024-04-02T16:22:53.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95e2ffd640 1 -- 172.21.15.50:0/3205665785 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8031b30 con 0x7f95e40d2e50 2024-04-02T16:22:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 -- 172.21.15.50:0/3205665785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e40d2e50 msgr2=0x7f95e40d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 --2- 172.21.15.50:0/3205665785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e40d2e50 0x7f95e40d0cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097d0 tx=0x7f95d802b810 comp rx=0 tx=0).stop 2024-04-02T16:22:53.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 -- 172.21.15.50:0/3205665785 shutdown_connections 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 --2- 172.21.15.50:0/3205665785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e40d2e50 0x7f95e40d0cc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 --2- 172.21.15.50:0/3205665785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e4161c80 0x7f95e40d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 -- 172.21.15.50:0/3205665785 >> 172.21.15.50:0/3205665785 conn(0x7f95e409d5e0 msgr2=0x7f95e40d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 -- 172.21.15.50:0/3205665785 shutdown_connections 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.102+0000 7f95ea38d640 1 -- 172.21.15.50:0/3205665785 wait complete. 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.103+0000 7f95ea38d640 1 Processor -- start 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.103+0000 7f95ea38d640 1 -- start start 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95ea38d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e40d1200 0x7f95e40cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95ea38d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e40d2e50 0x7f95e40b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95ea38d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e4161c80 0x7f95e4150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95ea38d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f95e40b2e70 con 0x7f95e4161c80 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95ea38d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95e40d0390 con 0x7f95e40d2e50 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95ea38d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95e40b2ff0 con 0x7f95e40d1200 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95e892a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e4161c80 0x7f95e4150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95e892a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e4161c80 0x7f95e4150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38236/0 (socket says 172.21.15.50:38236) 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95e892a640 1 -- 172.21.15.50:0/2175101519 learned_addr learned my addr 172.21.15.50:0/2175101519 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95e37fe640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e40d2e50 0x7f95e40b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.104+0000 7f95e3fff640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e40d1200 0x7f95e40cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e892a640 1 -- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e40d1200 msgr2=0x7f95e40cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e892a640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e40d1200 0x7f95e40cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e892a640 1 -- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e40d2e50 msgr2=0x7f95e40b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e892a640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e40d2e50 0x7f95e40b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e892a640 1 -- 172.21.15.50:0/2175101519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e40b51b0 con 0x7f95e4161c80 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e3fff640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e40d1200 0x7f95e40cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e892a640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e4161c80 0x7f95e4150670 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c730 tx=0x7f95dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e0ff9640 1 -- 172.21.15.50:0/2175101519 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc018070 con 0x7f95e4161c80 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e0ff9640 1 -- 172.21.15.50:0/2175101519 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc00ce40 con 0x7f95e4161c80 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e40cb1c0 con 0x7f95e4161c80 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95e0ff9640 1 -- 172.21.15.50:0/2175101519 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00a860 con 0x7f95e4161c80 2024-04-02T16:22:53.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.105+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e40cdbc0 con 0x7f95e4161c80 2024-04-02T16:22:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.106+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e4075760 con 0x7f95e4161c80 2024-04-02T16:22:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.106+0000 7f95e0ff9640 1 -- 172.21.15.50:0/2175101519 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95dc01b070 con 0x7f95e4161c80 2024-04-02T16:22:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.107+0000 7f95e0ff9640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f95d006c920 0x7f95d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.107+0000 7f95e0ff9640 1 -- 172.21.15.50:0/2175101519 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f95dc08b3e0 con 0x7f95e4161c80 2024-04-02T16:22:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.107+0000 7f95e3fff640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f95d006c920 0x7f95d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.108+0000 7f95e3fff640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f95d006c920 0x7f95d006ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009b90 tx=0x7f95d4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.111+0000 7f95e0ff9640 1 -- 172.21.15.50:0/2175101519 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f95e40b51b0 con 0x7f95e4161c80 2024-04-02T16:22:53.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.296+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95e40ce840 con 0x7f95e4161c80 2024-04-02T16:22:53.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.297+0000 7f95e0ff9640 1 -- 172.21.15.50:0/2175101519 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f95dc058da0 con 0x7f95e4161c80 2024-04-02T16:22:53.298 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.298+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f95d006c920 msgr2=0x7f95d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.298+0000 7f95ea38d640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f95d006c920 0x7f95d006ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009b90 tx=0x7f95d4008040 comp rx=0 tx=0).stop 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.298+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e4161c80 msgr2=0x7f95e4150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e4161c80 0x7f95e4150670 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c730 tx=0x7f95dc00cbf0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 shutdown_connections 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f95d006c920 0x7f95d006ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95e4161c80 0x7f95e4150670 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e40d2e50 0x7f95e40b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 --2- 172.21.15.50:0/2175101519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e40d1200 0x7f95e40cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 >> 172.21.15.50:0/2175101519 conn(0x7f95e409d5e0 msgr2=0x7f95e4160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:53.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 shutdown_connections 2024-04-02T16:22:53.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.299+0000 7f95ea38d640 1 -- 172.21.15.50:0/2175101519 wait complete. 2024-04-02T16:22:53.311 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:53.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22e1d96640 1 Processor -- start 2024-04-02T16:22:53.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22e1d96640 1 -- start start 2024-04-02T16:22:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22e1d96640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc161c80 0x7f22dc0d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22e1d96640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f22dc0d13d0 0x7f22dc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22e1d96640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22dc0b10e0 con 0x7f22dc0d2e50 2024-04-02T16:22:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22e1d96640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f22dc0b1380 con 0x7f22dc0d13d0 2024-04-02T16:22:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22e1d96640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0b1660 con 0x7f22dc161c80 2024-04-02T16:22:53.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.415+0000 7f22dbfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f22dc0d13d0 0x7f22dc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22d2ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f22dc0d2e50 0x7f22dc0d0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34684/0 (socket says 172.21.15.50:34684) 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22d2ffd640 1 -- 172.21.15.50:0/2440018783 learned_addr learned my addr 172.21.15.50:0/2440018783 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22dbfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f22dc0d13d0 0x7f22dc0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38250/0 (socket says 172.21.15.50:38250) 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22db7fe640 1 --2- 172.21.15.50:0/2440018783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc161c80 0x7f22dc0d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22dbfff640 1 -- 172.21.15.50:0/2440018783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc161c80 msgr2=0x7f22dc0d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22dbfff640 1 --2- 172.21.15.50:0/2440018783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc161c80 0x7f22dc0d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22dbfff640 1 -- 172.21.15.50:0/2440018783 >> v1:172.21.15.50:6789/0 conn(0x7f22dc0d2e50 legacy=0x7f22dc0d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22dbfff640 1 -- 172.21.15.50:0/2440018783 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0d3340 con 0x7f22dc0d13d0 2024-04-02T16:22:53.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.416+0000 7f22daffd640 1 -- 172.21.15.50:0/2440018783 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472286045 0 0) 0x7f22dc0b10e0 con 0x7f22dc0d2e50 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22dbfff640 1 --2- 172.21.15.50:0/2440018783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d13d0 0x7f22dc0b1e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009d90 tx=0x7f22d402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6aa9db85c0a6d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22daffd640 1 -- 172.21.15.50:0/2440018783 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4039070 con 0x7f22dc0d13d0 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22daffd640 1 -- 172.21.15.50:0/2440018783 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d402cbc0 con 0x7f22dc0d13d0 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22daffd640 1 -- 172.21.15.50:0/2440018783 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4034440 con 0x7f22dc0d13d0 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 -- 172.21.15.50:0/2440018783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d13d0 msgr2=0x7f22dc0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2440018783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d13d0 0x7f22dc0b1e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009d90 tx=0x7f22d402be10 comp rx=0 tx=0).stop 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 -- 172.21.15.50:0/2440018783 shutdown_connections 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2440018783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d13d0 0x7f22dc0b1e60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2440018783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc161c80 0x7f22dc0d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 -- 172.21.15.50:0/2440018783 >> 172.21.15.50:0/2440018783 conn(0x7f22dc09d5e0 msgr2=0x7f22dc0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 -- 172.21.15.50:0/2440018783 shutdown_connections 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.417+0000 7f22e1d96640 1 -- 172.21.15.50:0/2440018783 wait complete. 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.418+0000 7f22e1d96640 1 Processor -- start 2024-04-02T16:22:53.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22e1d96640 1 -- start start 2024-04-02T16:22:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22e1d96640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc0d13d0 0x7f22dc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22e1d96640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d2e50 0x7f22dc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22e1d96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc161c80 0x7f22dc1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22e1d96640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0b2520 con 0x7f22dc0d2e50 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22e1d96640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0b23a0 con 0x7f22dc161c80 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22e1d96640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0b26a0 con 0x7f22dc0d13d0 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22d2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d2e50 0x7f22dc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22d2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d2e50 0x7f22dc0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38256/0 (socket says 172.21.15.50:38256) 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.419+0000 7f22d2ffd640 1 -- 172.21.15.50:0/2989884793 learned_addr learned my addr 172.21.15.50:0/2989884793 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22db7fe640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc0d13d0 0x7f22dc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22dbfff640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc161c80 0x7f22dc1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d2ffd640 1 -- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc0d13d0 msgr2=0x7f22dc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d2ffd640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc0d13d0 0x7f22dc0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d2ffd640 1 -- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc161c80 msgr2=0x7f22dc1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d2ffd640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc161c80 0x7f22dc1f4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d2ffd640 1 -- 172.21.15.50:0/2989884793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0cd700 con 0x7f22dc0d2e50 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22db7fe640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc0d13d0 0x7f22dc0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d2ffd640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d2e50 0x7f22dc0cceb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c4e0 tx=0x7f22c400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d8ff9640 1 -- 172.21.15.50:0/2989884793 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4018070 con 0x7f22dc0d2e50 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d8ff9640 1 -- 172.21.15.50:0/2989884793 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4007500 con 0x7f22dc0d2e50 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc0cd9e0 con 0x7f22dc0d2e50 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22d8ff9640 1 -- 172.21.15.50:0/2989884793 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4004a20 con 0x7f22dc0d2e50 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.420+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc0cd3f0 con 0x7f22dc0d2e50 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.421+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc075760 con 0x7f22dc0d2e50 2024-04-02T16:22:53.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.421+0000 7f22d8ff9640 1 -- 172.21.15.50:0/2989884793 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22c401b070 con 0x7f22dc0d2e50 2024-04-02T16:22:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.422+0000 7f22d8ff9640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f22bc06c8f0 0x7f22bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.422+0000 7f22d8ff9640 1 -- 172.21.15.50:0/2989884793 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f22c408b280 con 0x7f22dc0d2e50 2024-04-02T16:22:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.422+0000 7f22db7fe640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f22bc06c8f0 0x7f22bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.423+0000 7f22db7fe640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f22bc06c8f0 0x7f22bc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.426+0000 7f22d8ff9640 1 -- 172.21.15.50:0/2989884793 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f22c4059350 con 0x7f22dc0d2e50 2024-04-02T16:22:53.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.613+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22dc0b9480 con 0x7f22dc0d2e50 2024-04-02T16:22:53.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.613+0000 7f22d8ff9640 1 -- 172.21.15.50:0/2989884793 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f22c4058c50 con 0x7f22dc0d2e50 2024-04-02T16:22:53.614 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:53.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f22bc06c8f0 msgr2=0x7f22bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f22bc06c8f0 0x7f22bc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).stop 2024-04-02T16:22:53.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d2e50 msgr2=0x7f22dc0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d2e50 0x7f22dc0cceb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c4e0 tx=0x7f22c400c9a0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 shutdown_connections 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc161c80 0x7f22dc1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f22bc06c8f0 0x7f22bc06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22dc0d2e50 0x7f22dc0cceb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 --2- 172.21.15.50:0/2989884793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22dc0d13d0 0x7f22dc0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 >> 172.21.15.50:0/2989884793 conn(0x7f22dc09d5e0 msgr2=0x7f22dc0b7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 shutdown_connections 2024-04-02T16:22:53.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.615+0000 7f22e1d96640 1 -- 172.21.15.50:0/2989884793 wait complete. 2024-04-02T16:22:53.627 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:53.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.730+0000 7f1ca7263640 1 Processor -- start 2024-04-02T16:22:53.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.730+0000 7f1ca7263640 1 -- start start 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1ca7263640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ca0141500 0x7f1ca0140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1ca7263640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca00cf450 con 0x7f1ca013e120 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1ca7263640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca00cf730 con 0x7f1ca0140e60 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1ca7263640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca01412b0 con 0x7f1ca0141500 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1ca4fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1ca013e120 0x7f1ca00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34692/0 (socket says 172.21.15.50:34692) 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1ca4fff640 1 -- 172.21.15.50:0/1137803376 learned_addr learned my addr 172.21.15.50:0/1137803376 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1c9ffff640 1 --2- 172.21.15.50:0/1137803376 >> v2:172.21.15.188:3300/0 conn(0x7f1ca0141500 0x7f1ca0140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893519686 0 0) 0x7f1ca00cf450 con 0x7f1ca013e120 2024-04-02T16:22:53.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.731+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c90003520 con 0x7f1ca013e120 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3496611507 0 0) 0x7f1c90003520 con 0x7f1ca013e120 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 >> v2:172.21.15.188:3300/0 conn(0x7f1ca0141500 msgr2=0x7f1ca0140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 --2- 172.21.15.50:0/1137803376 >> v2:172.21.15.188:3300/0 conn(0x7f1ca0141500 0x7f1ca0140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 >> v1:172.21.15.188:6789/0 conn(0x7f1ca0140e60 legacy=0x7f1ca0071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca00cfa10 con 0x7f1ca013e120 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f1c94001670 con 0x7f1ca013e120 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 >> v1:172.21.15.50:6789/0 conn(0x7f1ca013e120 legacy=0x7f1ca00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c900039b0 0x7f1c90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c90004800 0x7f1c90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1c9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca01412b0 con 0x7f1c900051c0 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9000c060 con 0x7f1c90004800 2024-04-02T16:22:53.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9000c260 con 0x7f1c900039b0 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1ca5800640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1c9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1ca4fff640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c900039b0 0x7f1c90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1c9ffff640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c90004800 0x7f1c90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1ca5800640 1 -- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c900039b0 msgr2=0x7f1c90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1ca5800640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c900039b0 0x7f1c90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1ca5800640 1 -- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c90004800 msgr2=0x7f1c90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1ca5800640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c90004800 0x7f1c90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.732+0000 7f1ca5800640 1 -- 172.21.15.50:0/1137803376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c90003520 con 0x7f1c900051c0 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.733+0000 7f1ca5800640 1 -- 172.21.15.50:0/1137803376 reap_dead start 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.733+0000 7f1ca5800640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1c9000b9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c98001e50 tx=0x7f1c90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c6c6880b0e4d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.733+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98001a70 con 0x7f1c900051c0 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.733+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98003bb0 con 0x7f1c900051c0 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.733+0000 7f1c9f7fe640 1 -- 172.21.15.50:0/1137803376 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980050d0 con 0x7f1c900051c0 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.734+0000 7f1ca7263640 1 -- 172.21.15.50:0/1137803376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 msgr2=0x7f1c9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.734+0000 7f1ca7263640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1c9000b9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c98001e50 tx=0x7f1c90005830 comp rx=0 tx=0).stop 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.734+0000 7f1ca7263640 1 -- 172.21.15.50:0/1137803376 shutdown_connections 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.734+0000 7f1ca7263640 1 --2- 172.21.15.50:0/1137803376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1c9000b9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.734+0000 7f1ca7263640 1 -- 172.21.15.50:0/1137803376 >> 172.21.15.50:0/1137803376 conn(0x7f1ca006eb90 msgr2=0x7f1ca006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.734+0000 7f1ca7263640 1 -- 172.21.15.50:0/1137803376 shutdown_connections 2024-04-02T16:22:53.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.734+0000 7f1ca7263640 1 -- 172.21.15.50:0/1137803376 wait complete. 2024-04-02T16:22:53.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.735+0000 7f1ca7263640 1 Processor -- start 2024-04-02T16:22:53.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.735+0000 7f1ca7263640 1 -- start start 2024-04-02T16:22:53.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca7263640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1ca0155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca7263640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ca012cbc0 0x7f1ca0155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca7263640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ca0143580 0x7f1ca0143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca7263640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca013e500 con 0x7f1c900051c0 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca7263640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0071400 con 0x7f1ca012cbc0 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca7263640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca0140920 con 0x7f1ca0143580 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1ca0155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1ca0155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38278/0 (socket says 172.21.15.50:38278) 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 -- 172.21.15.50:0/12596908 learned_addr learned my addr 172.21.15.50:0/12596908 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1c9ffff640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ca012cbc0 0x7f1ca0155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca5800640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ca0143580 0x7f1ca0143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 -- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ca0143580 msgr2=0x7f1ca0143a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ca0143580 0x7f1ca0143a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 -- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ca012cbc0 msgr2=0x7f1ca0155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ca012cbc0 0x7f1ca0155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 -- 172.21.15.50:0/12596908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca0155550 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca5800640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ca0143580 0x7f1ca0143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.736+0000 7f1ca4fff640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1ca0155e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1c940072c0 tx=0x7f1c94007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.737+0000 7f1c9d7fa640 1 -- 172.21.15.50:0/12596908 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400f8e0 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.737+0000 7f1c9d7fa640 1 -- 172.21.15.50:0/12596908 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c94018070 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.737+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca0156420 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.737+0000 7f1c9d7fa640 1 -- 172.21.15.50:0/12596908 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400fce0 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.737+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca01569e0 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.738+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca0075760 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.738+0000 7f1c9d7fa640 1 -- 172.21.15.50:0/12596908 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c9401d460 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.738+0000 7f1c9d7fa640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1c8406c8f0 0x7f1c8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.738+0000 7f1c9d7fa640 1 -- 172.21.15.50:0/12596908 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f1c9408c050 con 0x7f1c900051c0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.739+0000 7f1c9ffff640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1c8406c8f0 0x7f1c8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:53.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.739+0000 7f1c9ffff640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1c8406c8f0 0x7f1c8406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0072890 tx=0x7f1c8c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:53.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.743+0000 7f1c9d7fa640 1 -- 172.21.15.50:0/12596908 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1c9405a060 con 0x7f1c900051c0 2024-04-02T16:22:53.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.927+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1ca00734f0 con 0x7f1c900051c0 2024-04-02T16:22:53.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.927+0000 7f1c9d7fa640 1 -- 172.21.15.50:0/12596908 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f1c94059960 con 0x7f1c900051c0 2024-04-02T16:22:53.929 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:53.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1c8406c8f0 msgr2=0x7f1c8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1c8406c8f0 0x7f1c8406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0072890 tx=0x7f1c8c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 msgr2=0x7f1ca0155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1ca0155e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1c940072c0 tx=0x7f1c94007780 comp rx=0 tx=0).stop 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 shutdown_connections 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ca0143580 0x7f1ca0143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ca012cbc0 0x7f1ca0155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1c8406c8f0 0x7f1c8406edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 --2- 172.21.15.50:0/12596908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c900051c0 0x7f1ca0155e40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 >> 172.21.15.50:0/12596908 conn(0x7f1ca006eb90 msgr2=0x7f1ca015ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:53.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 shutdown_connections 2024-04-02T16:22:53.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:53.929+0000 7f1ca7263640 1 -- 172.21.15.50:0/12596908 wait complete. 2024-04-02T16:22:53.942 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:22:54.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.043+0000 7f4be5241640 1 Processor -- start 2024-04-02T16:22:54.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.043+0000 7f4be5241640 1 -- start start 2024-04-02T16:22:54.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4be5241640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4be0144bd0 0x7f4be0072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4be5241640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be0073350 con 0x7f4be00b58b0 2024-04-02T16:22:54.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4be5241640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be0072ab0 con 0x7f4be00b33c0 2024-04-02T16:22:54.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4be5241640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f4be00ce760 con 0x7f4be0144bd0 2024-04-02T16:22:54.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bded9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4be0144bd0 0x7f4be0072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bded9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f4be0144bd0 0x7f4be0072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38294/0 (socket says 172.21.15.50:38294) 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bded9d640 1 -- 172.21.15.50:0/3056395033 learned_addr learned my addr 172.21.15.50:0/3056395033 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bde59c640 1 --1- 172.21.15.50:0/3056395033 >> v1:172.21.15.50:6789/0 conn(0x7f4be00b58b0 0x7f4be00b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34704/0 (socket says 172.21.15.50:34704) 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bded9d640 1 -- 172.21.15.50:0/3056395033 >> v1:172.21.15.188:6789/0 conn(0x7f4be00b33c0 legacy=0x7f4be00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bded9d640 1 -- 172.21.15.50:0/3056395033 >> v1:172.21.15.50:6789/0 conn(0x7f4be00b58b0 legacy=0x7f4be00b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bded9d640 1 -- 172.21.15.50:0/3056395033 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be0072d90 con 0x7f4be0144bd0 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.044+0000 7f4bddd9b640 1 -- 172.21.15.50:0/3056395033 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215877392 0 0) 0x7f4bc8001110 con 0x7f4be00b58b0 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.045+0000 7f4bded9d640 1 --2- 172.21.15.50:0/3056395033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be0072340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009870 tx=0x7f4bd802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de8e028d0072dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.045+0000 7f4bddd9b640 1 -- 172.21.15.50:0/3056395033 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8034070 con 0x7f4be0144bd0 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.045+0000 7f4bddd9b640 1 -- 172.21.15.50:0/3056395033 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd8030860 con 0x7f4be0144bd0 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.045+0000 7f4bddd9b640 1 -- 172.21.15.50:0/3056395033 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8030c30 con 0x7f4be0144bd0 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.046+0000 7f4be5241640 1 -- 172.21.15.50:0/3056395033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 msgr2=0x7f4be0072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.046+0000 7f4be5241640 1 --2- 172.21.15.50:0/3056395033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be0072340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009870 tx=0x7f4bd802b9a0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.046+0000 7f4be5241640 1 -- 172.21.15.50:0/3056395033 shutdown_connections 2024-04-02T16:22:54.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.046+0000 7f4be5241640 1 --2- 172.21.15.50:0/3056395033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be0072340 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.046+0000 7f4be5241640 1 -- 172.21.15.50:0/3056395033 >> 172.21.15.50:0/3056395033 conn(0x7f4be006e6c0 msgr2=0x7f4be006eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:54.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.046+0000 7f4be5241640 1 -- 172.21.15.50:0/3056395033 shutdown_connections 2024-04-02T16:22:54.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.046+0000 7f4be5241640 1 -- 172.21.15.50:0/3056395033 wait complete. 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.047+0000 7f4be5241640 1 Processor -- start 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.047+0000 7f4be5241640 1 -- start start 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4be5241640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4be00b33c0 0x7f4be01566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4be5241640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be00b58b0 0x7f4be0140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4be5241640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be013e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4be5241640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00ce990 con 0x7f4be0144bd0 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4be5241640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00b5d10 con 0x7f4be00b33c0 2024-04-02T16:22:54.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4be5241640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4be00cf5c0 con 0x7f4be00b58b0 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be013e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be013e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38302/0 (socket says 172.21.15.50:38302) 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 -- 172.21.15.50:0/234528826 learned_addr learned my addr 172.21.15.50:0/234528826 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bded9d640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4be00b33c0 0x7f4be01566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bde59c640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be00b58b0 0x7f4be0140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 -- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be00b58b0 msgr2=0x7f4be0140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be00b58b0 0x7f4be0140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 -- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4be00b33c0 msgr2=0x7f4be01566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4be00b33c0 0x7f4be01566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bdf59e640 1 -- 172.21.15.50:0/234528826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be013fd10 con 0x7f4be0144bd0 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.048+0000 7f4bded9d640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4be00b33c0 0x7f4be01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.049+0000 7f4bdf59e640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be013e770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c6d0 tx=0x7f4bd000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.049+0000 7f4bd77fe640 1 -- 172.21.15.50:0/234528826 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0016020 con 0x7f4be0144bd0 2024-04-02T16:22:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.049+0000 7f4bd77fe640 1 -- 172.21.15.50:0/234528826 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd000a840 con 0x7f4be0144bd0 2024-04-02T16:22:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.049+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be013fff0 con 0x7f4be0144bd0 2024-04-02T16:22:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.049+0000 7f4bd77fe640 1 -- 172.21.15.50:0/234528826 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000ac70 con 0x7f4be0144bd0 2024-04-02T16:22:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.049+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be0156be0 con 0x7f4be0144bd0 2024-04-02T16:22:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.050+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be0074a10 con 0x7f4be0144bd0 2024-04-02T16:22:54.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.050+0000 7f4bd77fe640 1 -- 172.21.15.50:0/234528826 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bd0019020 con 0x7f4be0144bd0 2024-04-02T16:22:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.050+0000 7f4bd77fe640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4bc006c8f0 0x7f4bc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.051+0000 7f4bd77fe640 1 -- 172.21.15.50:0/234528826 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f4bd008c4b0 con 0x7f4be0144bd0 2024-04-02T16:22:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.051+0000 7f4bded9d640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4bc006c8f0 0x7f4bc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.051+0000 7f4bded9d640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4bc006c8f0 0x7f4bc006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8036000 tx=0x7f4bd803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.055+0000 7f4bd77fe640 1 -- 172.21.15.50:0/234528826 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4bd0020020 con 0x7f4be0144bd0 2024-04-02T16:22:54.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.250+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4be0075760 con 0x7f4be0144bd0 2024-04-02T16:22:54.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.250+0000 7f4bd77fe640 1 -- 172.21.15.50:0/234528826 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f4bd005a440 con 0x7f4be0144bd0 2024-04-02T16:22:54.252 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4bc006c8f0 msgr2=0x7f4bc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4bc006c8f0 0x7f4bc006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8036000 tx=0x7f4bd803b040 comp rx=0 tx=0).stop 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 msgr2=0x7f4be013e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be013e770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c6d0 tx=0x7f4bd000cb90 comp rx=0 tx=0).stop 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 shutdown_connections 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f4bc006c8f0 0x7f4bc006edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4be0144bd0 0x7f4be013e770 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be00b58b0 0x7f4be0140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 --2- 172.21.15.50:0/234528826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4be00b33c0 0x7f4be01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 >> 172.21.15.50:0/234528826 conn(0x7f4be006e6c0 msgr2=0x7f4be00b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:54.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 shutdown_connections 2024-04-02T16:22:54.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.252+0000 7f4be5241640 1 -- 172.21.15.50:0/234528826 wait complete. 2024-04-02T16:22:54.264 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-02T16:22:54.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:54.265 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-02T16:22:54.292 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:22:54.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:54.292 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T16:22:54.347 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:54.347 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:54.366 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:54.366 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:54.425 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:54.425 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:54.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.584+0000 7f40fd359640 1 Processor -- start 2024-04-02T16:22:54.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.584+0000 7f40fd359640 1 -- start start 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.584+0000 7f40fd359640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f40f815bfe0 0x7f40f8165f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.584+0000 7f40fd359640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f815faf0 con 0x7f40f813deb0 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.584+0000 7f40fd359640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f815ed30 con 0x7f40f80b22d0 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.584+0000 7f40fd359640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f40f815f010 con 0x7f40f815bfe0 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f6ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f40f813deb0 0x7f40f80b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34720/0 (socket says 172.21.15.50:34720) 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f77fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f40f815bfe0 0x7f40f8165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f6ffd640 1 -- 172.21.15.50:0/2955532816 learned_addr learned my addr 172.21.15.50:0/2955532816 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:54.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f67fc640 1 -- 172.21.15.50:0/2955532816 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849163984 0 0) 0x7f40f815faf0 con 0x7f40f813deb0 2024-04-02T16:22:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f67fc640 1 -- 172.21.15.50:0/2955532816 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40e4003520 con 0x7f40f813deb0 2024-04-02T16:22:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f77fe640 1 -- 172.21.15.50:0/2955532816 >> v1:172.21.15.188:6789/0 conn(0x7f40f80b22d0 legacy=0x7f40f815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f77fe640 1 -- 172.21.15.50:0/2955532816 >> v1:172.21.15.50:6789/0 conn(0x7f40f813deb0 legacy=0x7f40f80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.585+0000 7f40f77fe640 1 -- 172.21.15.50:0/2955532816 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f815f2f0 con 0x7f40f815bfe0 2024-04-02T16:22:54.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40f77fe640 1 --2- 172.21.15.50:0/2955532816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f8165f40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d90 tx=0x7f40f002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d9223a814b424e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40f67fc640 1 -- 172.21.15.50:0/2955532816 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0039070 con 0x7f40f815bfe0 2024-04-02T16:22:54.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40f67fc640 1 -- 172.21.15.50:0/2955532816 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f002cd00 con 0x7f40f815bfe0 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40f67fc640 1 -- 172.21.15.50:0/2955532816 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0034660 con 0x7f40f815bfe0 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40fd359640 1 -- 172.21.15.50:0/2955532816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 msgr2=0x7f40f8165f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40fd359640 1 --2- 172.21.15.50:0/2955532816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f8165f40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d90 tx=0x7f40f002bf50 comp rx=0 tx=0).stop 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40fd359640 1 -- 172.21.15.50:0/2955532816 shutdown_connections 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40fd359640 1 --2- 172.21.15.50:0/2955532816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f8165f40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40fd359640 1 -- 172.21.15.50:0/2955532816 >> 172.21.15.50:0/2955532816 conn(0x7f40f806ebf0 msgr2=0x7f40f806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40fd359640 1 -- 172.21.15.50:0/2955532816 shutdown_connections 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.586+0000 7f40fd359640 1 -- 172.21.15.50:0/2955532816 wait complete. 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.587+0000 7f40fd359640 1 Processor -- start 2024-04-02T16:22:54.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40fd359640 1 -- start start 2024-04-02T16:22:54.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40fd359640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f80b22d0 0x7f40f813a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40fd359640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40f813deb0 0x7f40f8137040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40fd359640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f81545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40fd359640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40f813e410 con 0x7f40f815bfe0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40fd359640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40f813e290 con 0x7f40f80b22d0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40fd359640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40f80cf8a0 con 0x7f40f813deb0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f81545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f81545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38326/0 (socket says 172.21.15.50:38326) 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40f77fe640 1 -- 172.21.15.50:0/3059859027 learned_addr learned my addr 172.21.15.50:0/3059859027 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40f6ffd640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f80b22d0 0x7f40f813a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.588+0000 7f40effff640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40f813deb0 0x7f40f8137040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40f77fe640 1 -- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40f813deb0 msgr2=0x7f40f8137040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40f77fe640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40f813deb0 0x7f40f8137040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40f77fe640 1 -- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f80b22d0 msgr2=0x7f40f813a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40f77fe640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f80b22d0 0x7f40f813a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40f77fe640 1 -- 172.21.15.50:0/3059859027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f81f4640 con 0x7f40f815bfe0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40f77fe640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f81545d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d90 tx=0x7f40f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40ef7fe640 1 -- 172.21.15.50:0/3059859027 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0039050 con 0x7f40f815bfe0 2024-04-02T16:22:54.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40ef7fe640 1 -- 172.21.15.50:0/3059859027 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f0045070 con 0x7f40f815bfe0 2024-04-02T16:22:54.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f8138730 con 0x7f40f815bfe0 2024-04-02T16:22:54.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40ef7fe640 1 -- 172.21.15.50:0/3059859027 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0009450 con 0x7f40f815bfe0 2024-04-02T16:22:54.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.589+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f8138170 con 0x7f40f815bfe0 2024-04-02T16:22:54.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.590+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f80756f0 con 0x7f40f815bfe0 2024-04-02T16:22:54.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.591+0000 7f40ef7fe640 1 -- 172.21.15.50:0/3059859027 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40f003e850 con 0x7f40f815bfe0 2024-04-02T16:22:54.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.591+0000 7f40ef7fe640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40d806c8f0 0x7f40d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.591+0000 7f40ef7fe640 1 -- 172.21.15.50:0/3059859027 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f40f00aec60 con 0x7f40f815bfe0 2024-04-02T16:22:54.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.591+0000 7f40f6ffd640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40d806c8f0 0x7f40d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.592+0000 7f40f6ffd640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40d806c8f0 0x7f40d806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f40e80011e0 tx=0x7f40e8007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.595+0000 7f40ef7fe640 1 -- 172.21.15.50:0/3059859027 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f40f8138730 con 0x7f40f815bfe0 2024-04-02T16:22:54.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.784+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40f806c210 con 0x7f40f815bfe0 2024-04-02T16:22:54.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.785+0000 7f40ef7fe640 1 -- 172.21.15.50:0/3059859027 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7f40f81f4640 con 0x7f40f815bfe0 2024-04-02T16:22:54.786 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:54.786 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40d806c8f0 msgr2=0x7f40d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40d806c8f0 0x7f40d806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f40e80011e0 tx=0x7f40e8007d30 comp rx=0 tx=0).stop 2024-04-02T16:22:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 msgr2=0x7f40f81545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f81545d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d90 tx=0x7f40f0004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 shutdown_connections 2024-04-02T16:22:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40d806c8f0 0x7f40d806edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40f815bfe0 0x7f40f81545d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40f813deb0 0x7f40f8137040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 --2- 172.21.15.50:0/3059859027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f80b22d0 0x7f40f813a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 >> 172.21.15.50:0/3059859027 conn(0x7f40f806ebf0 msgr2=0x7f40f815dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 shutdown_connections 2024-04-02T16:22:54.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.787+0000 7f40fd359640 1 -- 172.21.15.50:0/3059859027 wait complete. 2024-04-02T16:22:54.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:54.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.903+0000 7fc9e38f4640 1 Processor -- start 2024-04-02T16:22:54.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.903+0000 7fc9e38f4640 1 -- start start 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.903+0000 7fc9e38f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc0d4980 0x7fc9dc162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.903+0000 7fc9e38f4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9dc0d0c80 0x7fc9dc0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.903+0000 7fc9e38f4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9dc0d5000 con 0x7fc9dc161ca0 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e38f4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fc9dc0d4de0 con 0x7fc9dc0d0c80 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e38f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc9dc1630f0 con 0x7fc9dc0d4980 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e1690640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc9dc161ca0 0x7fc9dc0cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34730/0 (socket says 172.21.15.50:34730) 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e1e91640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc9dc0d0c80 0x7fc9dc0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e1690640 1 -- 172.21.15.50:0/1938457576 learned_addr learned my addr 172.21.15.50:0/1938457576 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e0e8f640 1 --2- 172.21.15.50:0/1938457576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc0d4980 0x7fc9dc162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e1e91640 1 -- 172.21.15.50:0/1938457576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc0d4980 msgr2=0x7fc9dc162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e1e91640 1 --2- 172.21.15.50:0/1938457576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc0d4980 0x7fc9dc162bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e1e91640 1 -- 172.21.15.50:0/1938457576 >> v1:172.21.15.50:6789/0 conn(0x7fc9dc161ca0 legacy=0x7fc9dc0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9e1e91640 1 -- 172.21.15.50:0/1938457576 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc0cf0d0 con 0x7fc9dc0d0c80 2024-04-02T16:22:54.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.904+0000 7fc9dbfff640 1 -- 172.21.15.50:0/1938457576 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345247300 0 0) 0x7fc9dc0d5000 con 0x7fc9dc161ca0 2024-04-02T16:22:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e1e91640 1 --2- 172.21.15.50:0/1938457576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d0c80 0x7fc9dc0d10d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0097a0 tx=0x7fc9cc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622985209f21c9d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9dbfff640 1 -- 172.21.15.50:0/1938457576 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc038070 con 0x7fc9dc0d0c80 2024-04-02T16:22:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9dbfff640 1 -- 172.21.15.50:0/1938457576 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9cc031700 con 0x7fc9dc0d0c80 2024-04-02T16:22:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9dbfff640 1 -- 172.21.15.50:0/1938457576 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc031ad0 con 0x7fc9dc0d0c80 2024-04-02T16:22:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e38f4640 1 -- 172.21.15.50:0/1938457576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d0c80 msgr2=0x7fc9dc0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/1938457576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d0c80 0x7fc9dc0d10d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0097a0 tx=0x7fc9cc02b7a0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e38f4640 1 -- 172.21.15.50:0/1938457576 shutdown_connections 2024-04-02T16:22:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/1938457576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d0c80 0x7fc9dc0d10d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/1938457576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc0d4980 0x7fc9dc162bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e38f4640 1 -- 172.21.15.50:0/1938457576 >> 172.21.15.50:0/1938457576 conn(0x7fc9dc09d5c0 msgr2=0x7fc9dc0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.905+0000 7fc9e38f4640 1 -- 172.21.15.50:0/1938457576 shutdown_connections 2024-04-02T16:22:54.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.906+0000 7fc9e38f4640 1 -- 172.21.15.50:0/1938457576 wait complete. 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 Processor -- start 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 -- start start 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9dc0d0c80 0x7fc9dc0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d4980 0x7fc9dc1f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc161ca0 0x7fc9dc0b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc0cf690 con 0x7fc9dc0d4980 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc0d1610 con 0x7fc9dc0d0c80 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e38f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc9dc0ced80 con 0x7fc9dc161ca0 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e0e8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d4980 0x7fc9dc1f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e0e8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d4980 0x7fc9dc1f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38348/0 (socket says 172.21.15.50:38348) 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.907+0000 7fc9e0e8f640 1 -- 172.21.15.50:0/3308575637 learned_addr learned my addr 172.21.15.50:0/3308575637 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:54.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e1e91640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc161ca0 0x7fc9dc0b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e1690640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9dc0d0c80 0x7fc9dc0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e0e8f640 1 -- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc161ca0 msgr2=0x7fc9dc0b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e0e8f640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc161ca0 0x7fc9dc0b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e0e8f640 1 -- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9dc0d0c80 msgr2=0x7fc9dc0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e0e8f640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9dc0d0c80 0x7fc9dc0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e0e8f640 1 -- 172.21.15.50:0/3308575637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc0c82e0 con 0x7fc9dc0d4980 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e0e8f640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d4980 0x7fc9dc1f3ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000d5f0 tx=0x7fc9d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9d9ffb640 1 -- 172.21.15.50:0/3308575637 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00125f0 con 0x7fc9dc0d4980 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9d9ffb640 1 -- 172.21.15.50:0/3308575637 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d0019070 con 0x7fc9dc0d4980 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9dc0c8000 con 0x7fc9dc0d4980 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9d9ffb640 1 -- 172.21.15.50:0/3308575637 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00129d0 con 0x7fc9dc0d4980 2024-04-02T16:22:54.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.908+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9dc0cb7c0 con 0x7fc9dc0d4980 2024-04-02T16:22:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.909+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9dc075760 con 0x7fc9dc0d4980 2024-04-02T16:22:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.909+0000 7fc9d9ffb640 1 -- 172.21.15.50:0/3308575637 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9d0012b70 con 0x7fc9dc0d4980 2024-04-02T16:22:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.910+0000 7fc9d9ffb640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc9c806c8f0 0x7fc9c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.910+0000 7fc9d9ffb640 1 -- 172.21.15.50:0/3308575637 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fc9d008c190 con 0x7fc9dc0d4980 2024-04-02T16:22:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.910+0000 7fc9e1690640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc9c806c8f0 0x7fc9c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:54.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.911+0000 7fc9e1690640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc9c806c8f0 0x7fc9c806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40012e0 tx=0x7fc9d4001700 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:54.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:54.914+0000 7fc9d9ffb640 1 -- 172.21.15.50:0/3308575637 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc9dc0c82e0 con 0x7fc9dc0d4980 2024-04-02T16:22:55.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.096+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9dc0d0a60 con 0x7fc9dc0d4980 2024-04-02T16:22:55.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.097+0000 7fc9d9ffb640 1 -- 172.21.15.50:0/3308575637 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fc9d0059be0 con 0x7fc9dc0d4980 2024-04-02T16:22:55.098 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:55.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc9c806c8f0 msgr2=0x7fc9c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc9c806c8f0 0x7fc9c806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40012e0 tx=0x7fc9d4001700 comp rx=0 tx=0).stop 2024-04-02T16:22:55.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d4980 msgr2=0x7fc9dc1f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d4980 0x7fc9dc1f3ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000d5f0 tx=0x7fc9d000dab0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 shutdown_connections 2024-04-02T16:22:55.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9dc161ca0 0x7fc9dc0b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc9c806c8f0 0x7fc9c806edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9dc0d4980 0x7fc9dc1f3ab0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 --2- 172.21.15.50:0/3308575637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9dc0d0c80 0x7fc9dc0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 >> 172.21.15.50:0/3308575637 conn(0x7fc9dc09d5c0 msgr2=0x7fc9dc0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:55.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 shutdown_connections 2024-04-02T16:22:55.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.099+0000 7fc9e38f4640 1 -- 172.21.15.50:0/3308575637 wait complete. 2024-04-02T16:22:55.111 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:55.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.214+0000 7fb40d23a640 1 Processor -- start 2024-04-02T16:22:55.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.214+0000 7fb40d23a640 1 -- start start 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40d23a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb408162600 0x7fb4080b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40d23a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4081629e0 0x7fb4080cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40d23a640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40809b450 con 0x7fb408162f30 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40d23a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb4080d0aa0 con 0x7fb408162600 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40d23a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb40809b6f0 con 0x7fb4081629e0 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb406d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb408162600 0x7fb4080b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40659c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb408162f30 0x7fb4080d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34736/0 (socket says 172.21.15.50:34736) 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb406d9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb408162600 0x7fb4080b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38354/0 (socket says 172.21.15.50:38354) 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40659c640 1 -- 172.21.15.50:0/2433503766 learned_addr learned my addr 172.21.15.50:0/2433503766 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.215+0000 7fb40759e640 1 --2- 172.21.15.50:0/2433503766 >> v2:172.21.15.188:3300/0 conn(0x7fb4081629e0 0x7fb4080cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb406d9d640 1 -- 172.21.15.50:0/2433503766 >> v2:172.21.15.188:3300/0 conn(0x7fb4081629e0 msgr2=0x7fb4080cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb406d9d640 1 --2- 172.21.15.50:0/2433503766 >> v2:172.21.15.188:3300/0 conn(0x7fb4081629e0 0x7fb4080cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb406d9d640 1 -- 172.21.15.50:0/2433503766 >> v1:172.21.15.50:6789/0 conn(0x7fb408162f30 legacy=0x7fb4080d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb406d9d640 1 -- 172.21.15.50:0/2433503766 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40809b9d0 con 0x7fb408162600 2024-04-02T16:22:55.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb405d9b640 1 -- 172.21.15.50:0/2433503766 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3972515341 0 0) 0x7fb40809b450 con 0x7fb408162f30 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb406d9d640 1 --2- 172.21.15.50:0/2433503766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb408162600 0x7fb4080b4740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097a0 tx=0x7fb40002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ca10729ed6c254 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb405d9b640 1 -- 172.21.15.50:0/2433503766 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000315a0 con 0x7fb408162600 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb405d9b640 1 -- 172.21.15.50:0/2433503766 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb400037070 con 0x7fb408162600 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.216+0000 7fb405d9b640 1 -- 172.21.15.50:0/2433503766 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400031960 con 0x7fb408162600 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 -- 172.21.15.50:0/2433503766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb408162600 msgr2=0x7fb4080b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 --2- 172.21.15.50:0/2433503766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb408162600 0x7fb4080b4740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097a0 tx=0x7fb40002b840 comp rx=0 tx=0).stop 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 -- 172.21.15.50:0/2433503766 shutdown_connections 2024-04-02T16:22:55.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 --2- 172.21.15.50:0/2433503766 >> v2:172.21.15.188:3300/0 conn(0x7fb4081629e0 0x7fb4080cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 --2- 172.21.15.50:0/2433503766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb408162600 0x7fb4080b4740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 -- 172.21.15.50:0/2433503766 >> 172.21.15.50:0/2433503766 conn(0x7fb4080b3c20 msgr2=0x7fb4080b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 -- 172.21.15.50:0/2433503766 shutdown_connections 2024-04-02T16:22:55.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.217+0000 7fb40d23a640 1 -- 172.21.15.50:0/2433503766 wait complete. 2024-04-02T16:22:55.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.218+0000 7fb40d23a640 1 Processor -- start 2024-04-02T16:22:55.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.218+0000 7fb40d23a640 1 -- start start 2024-04-02T16:22:55.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40d23a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb408162600 0x7fb4080cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40d23a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4081629e0 0x7fb4080c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40d23a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb408162f30 0x7fb4081479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40d23a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4080d0350 con 0x7fb4081629e0 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40d23a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4080d1640 con 0x7fb408162f30 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40d23a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4080d04d0 con 0x7fb408162600 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4081629e0 0x7fb4080c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4081629e0 0x7fb4080c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38366/0 (socket says 172.21.15.50:38366) 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.219+0000 7fb40659c640 1 -- 172.21.15.50:0/1321139508 learned_addr learned my addr 172.21.15.50:0/1321139508 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb406d9d640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb408162600 0x7fb4080cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40759e640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb408162f30 0x7fb4081479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40659c640 1 -- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb408162600 msgr2=0x7fb4080cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40659c640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb408162600 0x7fb4080cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40659c640 1 -- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb408162f30 msgr2=0x7fb4081479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40659c640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb408162f30 0x7fb4081479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40659c640 1 -- 172.21.15.50:0/1321139508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4080c6d30 con 0x7fb4081629e0 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40659c640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4081629e0 0x7fb4080c7dd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c3e0 tx=0x7fb3f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb3ff7fe640 1 -- 172.21.15.50:0/1321139508 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0017070 con 0x7fb4081629e0 2024-04-02T16:22:55.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb3ff7fe640 1 -- 172.21.15.50:0/1321139508 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f0005ce0 con 0x7fb4081629e0 2024-04-02T16:22:55.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4080c7010 con 0x7fb4081629e0 2024-04-02T16:22:55.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb3ff7fe640 1 -- 172.21.15.50:0/1321139508 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0007310 con 0x7fb4081629e0 2024-04-02T16:22:55.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.220+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4080c75d0 con 0x7fb4081629e0 2024-04-02T16:22:55.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.221+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb408075760 con 0x7fb4081629e0 2024-04-02T16:22:55.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.222+0000 7fb3ff7fe640 1 -- 172.21.15.50:0/1321139508 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3f001a070 con 0x7fb4081629e0 2024-04-02T16:22:55.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.222+0000 7fb3ff7fe640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3e806c8f0 0x7fb3e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.222+0000 7fb3ff7fe640 1 -- 172.21.15.50:0/1321139508 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fb3f008b3a0 con 0x7fb4081629e0 2024-04-02T16:22:55.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.222+0000 7fb406d9d640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3e806c8f0 0x7fb3e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.223+0000 7fb406d9d640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3e806c8f0 0x7fb3e806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb400036000 tx=0x7fb40003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.227+0000 7fb3ff7fe640 1 -- 172.21.15.50:0/1321139508 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb3f00593b0 con 0x7fb4081629e0 2024-04-02T16:22:55.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.411+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4080cf9a0 con 0x7fb4081629e0 2024-04-02T16:22:55.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.411+0000 7fb3ff7fe640 1 -- 172.21.15.50:0/1321139508 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fb3f0058cb0 con 0x7fb4081629e0 2024-04-02T16:22:55.413 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:55.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3e806c8f0 msgr2=0x7fb3e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3e806c8f0 0x7fb3e806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb400036000 tx=0x7fb40003d040 comp rx=0 tx=0).stop 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4081629e0 msgr2=0x7fb4080c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4081629e0 0x7fb4080c7dd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c3e0 tx=0x7fb3f000c8a0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 shutdown_connections 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb408162f30 0x7fb4081479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3e806c8f0 0x7fb3e806edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4081629e0 0x7fb4080c7dd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 --2- 172.21.15.50:0/1321139508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb408162600 0x7fb4080cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 >> 172.21.15.50:0/1321139508 conn(0x7fb4080b3c20 msgr2=0x7fb4080ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 shutdown_connections 2024-04-02T16:22:55.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.413+0000 7fb40d23a640 1 -- 172.21.15.50:0/1321139508 wait complete. 2024-04-02T16:22:55.425 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:55.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.528+0000 7f34fc2dd640 1 Processor -- start 2024-04-02T16:22:55.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.528+0000 7f34fc2dd640 1 -- start start 2024-04-02T16:22:55.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.529+0000 7f34fc2dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 0x7f34f4070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.529+0000 7f34fc2dd640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f40d1e60 con 0x7f34f409aa40 2024-04-02T16:22:55.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.529+0000 7f34fc2dd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f40713a0 con 0x7f34f40d0660 2024-04-02T16:22:55.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.529+0000 7f34fc2dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34f40b19c0 con 0x7f34f40b4700 2024-04-02T16:22:55.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.529+0000 7f34fa079640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f34f409aa40 0x7f34f409ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:34750/0 (socket says 172.21.15.50:34750) 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.529+0000 7f34fa079640 1 -- 172.21.15.50:0/375131792 learned_addr learned my addr 172.21.15.50:0/375131792 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9878640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 0x7f34f4070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155762864 0 0) 0x7f34ec001110 con 0x7f34f409aa40 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34dc003520 con 0x7f34f409aa40 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2564548221 0 0) 0x7f34dc003520 con 0x7f34f409aa40 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 msgr2=0x7f34f4070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 0x7f34f4070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 >> v1:172.21.15.188:6789/0 conn(0x7f34f40d0660 legacy=0x7f34f40d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f4070de0 con 0x7f34f409aa40 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f34ec001840 con 0x7f34f409aa40 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 >> v1:172.21.15.50:6789/0 conn(0x7f34f409aa40 legacy=0x7f34f409ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 0x7f34dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc003f20 0x7f34dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34dc004d40 0x7f34dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc00b9f0 con 0x7f34dc003f20 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc00bba0 con 0x7f34dc004d40 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc00bda0 con 0x7f34f40b4700 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9878640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc003f20 0x7f34dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34fa079640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 0x7f34dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34fa87a640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34dc004d40 0x7f34dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9878640 1 -- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 msgr2=0x7f34dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9878640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34f40b4700 0x7f34dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9878640 1 -- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34dc004d40 msgr2=0x7f34dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9878640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34dc004d40 0x7f34dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34f9878640 1 -- 172.21.15.50:0/375131792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc003520 con 0x7f34dc003f20 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.530+0000 7f34fa87a640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34dc004d40 0x7f34dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.531+0000 7f34f9878640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc003f20 0x7f34dc004800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009a90 tx=0x7f34dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8926d39f0385e130 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.531+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8039070 con 0x7f34dc003f20 2024-04-02T16:22:55.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.531+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e802c5c0 con 0x7f34dc003f20 2024-04-02T16:22:55.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.531+0000 7f34f9077640 1 -- 172.21.15.50:0/375131792 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8009690 con 0x7f34dc003f20 2024-04-02T16:22:55.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.532+0000 7f34fc2dd640 1 -- 172.21.15.50:0/375131792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc003f20 msgr2=0x7f34dc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.532+0000 7f34fc2dd640 1 --2- 172.21.15.50:0/375131792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc003f20 0x7f34dc004800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009a90 tx=0x7f34dc005830 comp rx=0 tx=0).stop 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.532+0000 7f34fa87a640 1 -- 172.21.15.50:0/375131792 reap_dead start 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.532+0000 7f34fc2dd640 1 -- 172.21.15.50:0/375131792 shutdown_connections 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.532+0000 7f34fc2dd640 1 -- 172.21.15.50:0/375131792 >> 172.21.15.50:0/375131792 conn(0x7f34f40b3be0 msgr2=0x7f34f40b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.532+0000 7f34fc2dd640 1 -- 172.21.15.50:0/375131792 shutdown_connections 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.532+0000 7f34fc2dd640 1 -- 172.21.15.50:0/375131792 wait complete. 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.533+0000 7f34fc2dd640 1 Processor -- start 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.533+0000 7f34fc2dd640 1 -- start start 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34dc003f20 0x7f34f409f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34f409c920 0x7f34f415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34f40c9d70 0x7f34f40ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34f406c9b0 con 0x7f34f409c920 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34f406c6d0 con 0x7f34f40c9d70 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34f406cc90 con 0x7f34dc003f20 2024-04-02T16:22:55.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34f409c920 0x7f34f415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34f409c920 0x7f34f415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38374/0 (socket says 172.21.15.50:38374) 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 -- 172.21.15.50:0/532794226 learned_addr learned my addr 172.21.15.50:0/532794226 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 -- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34dc003f20 msgr2=0x7f34f409f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34dc003f20 0x7f34f409f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 -- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34f40c9d70 msgr2=0x7f34f40ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34f40c9d70 0x7f34f40ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 -- 172.21.15.50:0/532794226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f409cda0 con 0x7f34f409c920 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f9878640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34f409c920 0x7f34f415d4f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f34e8037040 tx=0x7f34e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f2ffd640 1 -- 172.21.15.50:0/532794226 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8039050 con 0x7f34f409c920 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f2ffd640 1 -- 172.21.15.50:0/532794226 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8045070 con 0x7f34f409c920 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f409d050 con 0x7f34f409c920 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34f2ffd640 1 -- 172.21.15.50:0/532794226 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e80044e0 con 0x7f34f409c920 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.534+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f409d610 con 0x7f34f409c920 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.535+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f4075760 con 0x7f34f409c920 2024-04-02T16:22:55.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.536+0000 7f34f2ffd640 1 -- 172.21.15.50:0/532794226 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34e8003850 con 0x7f34f409c920 2024-04-02T16:22:55.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.536+0000 7f34f2ffd640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34e006c8f0 0x7f34e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.536+0000 7f34f2ffd640 1 -- 172.21.15.50:0/532794226 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f34e80ad9e0 con 0x7f34f409c920 2024-04-02T16:22:55.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.536+0000 7f34fa079640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34e006c8f0 0x7f34e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.537+0000 7f34fa079640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34e006c8f0 0x7f34e006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0012b0 tx=0x7f34ec001320 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.541+0000 7f34f2ffd640 1 -- 172.21.15.50:0/532794226 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f34e804e020 con 0x7f34f409c920 2024-04-02T16:22:55.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.725+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34f40713a0 con 0x7f34f409c920 2024-04-02T16:22:55.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.725+0000 7f34f2ffd640 1 -- 172.21.15.50:0/532794226 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f34e807ba80 con 0x7f34f409c920 2024-04-02T16:22:55.726 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34e006c8f0 msgr2=0x7f34e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34e006c8f0 0x7f34e006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0012b0 tx=0x7f34ec001320 comp rx=0 tx=0).stop 2024-04-02T16:22:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34f409c920 msgr2=0x7f34f415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34f409c920 0x7f34f415d4f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f34e8037040 tx=0x7f34e8004730 comp rx=0 tx=0).stop 2024-04-02T16:22:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 shutdown_connections 2024-04-02T16:22:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34f40c9d70 0x7f34f40ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34e006c8f0 0x7f34e006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34f409c920 0x7f34f415d4f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 --2- 172.21.15.50:0/532794226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34dc003f20 0x7f34f409f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 >> 172.21.15.50:0/532794226 conn(0x7f34f40b3be0 msgr2=0x7f34f40afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:55.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 shutdown_connections 2024-04-02T16:22:55.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.727+0000 7f34fc2dd640 1 -- 172.21.15.50:0/532794226 wait complete. 2024-04-02T16:22:55.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:55.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.842+0000 7f91e7855640 1 Processor -- start 2024-04-02T16:22:55.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.842+0000 7f91e7855640 1 -- start start 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e7855640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00b1590 0x7f91e00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e7855640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f91e0070d30 0x7f91e009b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e7855640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e015c900 con 0x7f91e00d0830 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e7855640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f91e006f890 con 0x7f91e0070d30 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e7855640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91e00cd6e0 con 0x7f91e00b1590 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e55f1640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f91e00d0830 0x7f91e009aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47670/0 (socket says 172.21.15.50:47670) 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e55f1640 1 -- 172.21.15.50:0/1225422931 learned_addr learned my addr 172.21.15.50:0/1225422931 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e5df2640 1 --2- 172.21.15.50:0/1225422931 >> v2:172.21.15.50:3300/0 conn(0x7f91e0070d30 0x7f91e009b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e4df0640 1 --2- 172.21.15.50:0/1225422931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00b1590 0x7f91e00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91dffff640 1 -- 172.21.15.50:0/1225422931 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227781976 0 0) 0x7f91e015c900 con 0x7f91e00d0830 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91dffff640 1 -- 172.21.15.50:0/1225422931 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91c8003520 con 0x7f91e00d0830 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e5df2640 1 -- 172.21.15.50:0/1225422931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00b1590 msgr2=0x7f91e00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e5df2640 1 --2- 172.21.15.50:0/1225422931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00b1590 0x7f91e00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.843+0000 7f91e5df2640 1 -- 172.21.15.50:0/1225422931 >> v1:172.21.15.50:6789/0 conn(0x7f91e00d0830 legacy=0x7f91e009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.844+0000 7f91e5df2640 1 -- 172.21.15.50:0/1225422931 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e00cd9c0 con 0x7f91e0070d30 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.844+0000 7f91e5df2640 1 --2- 172.21.15.50:0/1225422931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e009b150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009d90 tx=0x7f91d002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f9a321a6a6b307 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.844+0000 7f91dffff640 1 -- 172.21.15.50:0/1225422931 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0039070 con 0x7f91e0070d30 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.844+0000 7f91dffff640 1 -- 172.21.15.50:0/1225422931 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d002cc50 con 0x7f91e0070d30 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.844+0000 7f91dffff640 1 -- 172.21.15.50:0/1225422931 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0034440 con 0x7f91e0070d30 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 -- 172.21.15.50:0/1225422931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 msgr2=0x7f91e009b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 --2- 172.21.15.50:0/1225422931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e009b150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009d90 tx=0x7f91d002bea0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 -- 172.21.15.50:0/1225422931 shutdown_connections 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 --2- 172.21.15.50:0/1225422931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e009b150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 --2- 172.21.15.50:0/1225422931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00b1590 0x7f91e00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 -- 172.21.15.50:0/1225422931 >> 172.21.15.50:0/1225422931 conn(0x7f91e00b3ad0 msgr2=0x7f91e00b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:55.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 -- 172.21.15.50:0/1225422931 shutdown_connections 2024-04-02T16:22:55.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.845+0000 7f91e7855640 1 -- 172.21.15.50:0/1225422931 wait complete. 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.846+0000 7f91e7855640 1 Processor -- start 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.846+0000 7f91e7855640 1 -- start start 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e7855640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e014b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e7855640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e00b1590 0x7f91e00af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e7855640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00d0830 0x7f91e00af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e7855640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91e006f520 con 0x7f91e0070d30 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e7855640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91e00cdf80 con 0x7f91e00b1590 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e7855640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91e009b690 con 0x7f91e00d0830 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e55f1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e55f1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e014b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36768/0 (socket says 172.21.15.50:36768) 2024-04-02T16:22:55.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e55f1640 1 -- 172.21.15.50:0/2345538189 learned_addr learned my addr 172.21.15.50:0/2345538189 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e4df0640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e00b1590 0x7f91e00af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e55f1640 1 -- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00d0830 msgr2=0x7f91e00af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e5df2640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00d0830 0x7f91e00af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e55f1640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00d0830 0x7f91e00af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e55f1640 1 -- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e00b1590 msgr2=0x7f91e00af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.847+0000 7f91e55f1640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e00b1590 0x7f91e00af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91e55f1640 1 -- 172.21.15.50:0/2345538189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e014b950 con 0x7f91e0070d30 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91e4df0640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e00b1590 0x7f91e00af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91e55f1640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e014b410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c3c0 tx=0x7f91d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91ddffb640 1 -- 172.21.15.50:0/2345538189 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8017070 con 0x7f91e0070d30 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91ddffb640 1 -- 172.21.15.50:0/2345538189 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d8005ce0 con 0x7f91e0070d30 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e01f7870 con 0x7f91e0070d30 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91ddffb640 1 -- 172.21.15.50:0/2345538189 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80072f0 con 0x7f91e0070d30 2024-04-02T16:22:55.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.848+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01f7e30 con 0x7f91e0070d30 2024-04-02T16:22:55.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.849+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e0075760 con 0x7f91e0070d30 2024-04-02T16:22:55.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.849+0000 7f91ddffb640 1 -- 172.21.15.50:0/2345538189 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91d801a070 con 0x7f91e0070d30 2024-04-02T16:22:55.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.850+0000 7f91ddffb640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f91cc06c920 0x7f91cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:55.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.850+0000 7f91ddffb640 1 -- 172.21.15.50:0/2345538189 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f91d808b2f0 con 0x7f91e0070d30 2024-04-02T16:22:55.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.850+0000 7f91e4df0640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f91cc06c920 0x7f91cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:55.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.851+0000 7f91e4df0640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f91cc06c920 0x7f91cc06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91e009ffe0 tx=0x7f91d4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:55.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:55.854+0000 7f91ddffb640 1 -- 172.21.15.50:0/2345538189 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f91e014b950 con 0x7f91e0070d30 2024-04-02T16:22:56.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.038+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f91e00d03a0 con 0x7f91e0070d30 2024-04-02T16:22:56.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.038+0000 7f91ddffb640 1 -- 172.21.15.50:0/2345538189 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f91d8059300 con 0x7f91e0070d30 2024-04-02T16:22:56.040 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:56.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f91cc06c920 msgr2=0x7f91cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f91cc06c920 0x7f91cc06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91e009ffe0 tx=0x7f91d4008040 comp rx=0 tx=0).stop 2024-04-02T16:22:56.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 msgr2=0x7f91e014b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e014b410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c3c0 tx=0x7f91d800c880 comp rx=0 tx=0).stop 2024-04-02T16:22:56.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 shutdown_connections 2024-04-02T16:22:56.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91e00d0830 0x7f91e00af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91e00b1590 0x7f91e00af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f91cc06c920 0x7f91cc06ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 --2- 172.21.15.50:0/2345538189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e0070d30 0x7f91e014b410 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 >> 172.21.15.50:0/2345538189 conn(0x7f91e00b3ad0 msgr2=0x7f91e00b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:56.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 shutdown_connections 2024-04-02T16:22:56.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.040+0000 7f91e7855640 1 -- 172.21.15.50:0/2345538189 wait complete. 2024-04-02T16:22:56.053 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:22:56.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.154+0000 7f21f2782640 1 Processor -- start 2024-04-02T16:22:56.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.154+0000 7f21f2782640 1 -- start start 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.155+0000 7f21f2782640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec071020 0x7f21ec09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.155+0000 7f21f2782640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21ec0d1440 0x7f21ec0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.155+0000 7f21f2782640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec06cc40 con 0x7f21ec09af80 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.155+0000 7f21f2782640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec09b3c0 con 0x7f21ec0d1440 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.155+0000 7f21f2782640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec06c960 con 0x7f21ec071020 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.155+0000 7f21f0d1f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21ec0d1440 0x7f21ec0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.155+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec071020 0x7f21ec09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21f0d1f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21ec0d1440 0x7f21ec0cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:54142/0 (socket says 172.21.15.50:54142) 2024-04-02T16:22:56.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21f0d1f640 1 -- 172.21.15.50:0/859349589 learned_addr learned my addr 172.21.15.50:0/859349589 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21eb7fe640 1 --1- 172.21.15.50:0/859349589 >> v1:172.21.15.188:6789/0 conn(0x7f21ec09af80 0x7f21ec0cd490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:60220/0 (socket says 172.21.15.50:60220) 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21f0d1f640 1 -- 172.21.15.50:0/859349589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec071020 msgr2=0x7f21ec09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21f0d1f640 1 --2- 172.21.15.50:0/859349589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec071020 0x7f21ec09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21f0d1f640 1 -- 172.21.15.50:0/859349589 >> v1:172.21.15.188:6789/0 conn(0x7f21ec09af80 legacy=0x7f21ec0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21f0d1f640 1 -- 172.21.15.50:0/859349589 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0cc6d0 con 0x7f21ec0d1440 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.156+0000 7f21eaffd640 1 -- 172.21.15.50:0/859349589 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978065788 0 0) 0x7f21ec06cc40 con 0x7f21ec09af80 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f0d1f640 1 --2- 172.21.15.50:0/859349589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 0x7f21ec0cdba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009e50 tx=0x7f21e402bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cedb1907e94b1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21eaffd640 1 -- 172.21.15.50:0/859349589 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e402cb90 con 0x7f21ec0d1440 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21eaffd640 1 -- 172.21.15.50:0/859349589 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e402cd30 con 0x7f21ec0d1440 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21eaffd640 1 -- 172.21.15.50:0/859349589 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4034610 con 0x7f21ec0d1440 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 -- 172.21.15.50:0/859349589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 msgr2=0x7f21ec0cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 --2- 172.21.15.50:0/859349589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 0x7f21ec0cdba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009e50 tx=0x7f21e402bff0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 -- 172.21.15.50:0/859349589 shutdown_connections 2024-04-02T16:22:56.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 --2- 172.21.15.50:0/859349589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 0x7f21ec0cdba0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 --2- 172.21.15.50:0/859349589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec071020 0x7f21ec09aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 -- 172.21.15.50:0/859349589 >> 172.21.15.50:0/859349589 conn(0x7f21ec0b3ad0 msgr2=0x7f21ec0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:56.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 -- 172.21.15.50:0/859349589 shutdown_connections 2024-04-02T16:22:56.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.157+0000 7f21f2782640 1 -- 172.21.15.50:0/859349589 wait complete. 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 Processor -- start 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 -- start start 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21ec071020 0x7f21ec0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec09af80 0x7f21ec1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 0x7f21ec0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0ccf70 con 0x7f21ec071020 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0ccc90 con 0x7f21ec0d1440 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.159+0000 7f21f2782640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec06db50 con 0x7f21ec09af80 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21ec071020 0x7f21ec0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21ec071020 0x7f21ec0c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36772/0 (socket says 172.21.15.50:36772) 2024-04-02T16:22:56.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 -- 172.21.15.50:0/2832335649 learned_addr learned my addr 172.21.15.50:0/2832335649 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21eb7fe640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec09af80 0x7f21ec1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21f0d1f640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 0x7f21ec0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 -- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec09af80 msgr2=0x7f21ec1f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec09af80 0x7f21ec1f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 -- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 msgr2=0x7f21ec0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 0x7f21ec0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 -- 172.21.15.50:0/2832335649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec153940 con 0x7f21ec071020 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21ebfff640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21ec071020 0x7f21ec0c90b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e870 tx=0x7f21e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21e8ff9640 1 -- 172.21.15.50:0/2832335649 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0017070 con 0x7f21ec071020 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21e8ff9640 1 -- 172.21.15.50:0/2832335649 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e000c940 con 0x7f21ec071020 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21e8ff9640 1 -- 172.21.15.50:0/2832335649 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000cd30 con 0x7f21ec071020 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec161af0 con 0x7f21ec071020 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.160+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec0cb950 con 0x7f21ec071020 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.161+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec075760 con 0x7f21ec071020 2024-04-02T16:22:56.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.162+0000 7f21e8ff9640 1 -- 172.21.15.50:0/2832335649 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21e001a070 con 0x7f21ec071020 2024-04-02T16:22:56.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.162+0000 7f21e8ff9640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.162+0000 7f21e8ff9640 1 -- 172.21.15.50:0/2832335649 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f21e008bd00 con 0x7f21ec071020 2024-04-02T16:22:56.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.163+0000 7f21eb7fe640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.163+0000 7f21eb7fe640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f21ec1f7270 tx=0x7f21dc007e40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:56.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.167+0000 7f21e8ff9640 1 -- 172.21.15.50:0/2832335649 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f21e0059750 con 0x7f21ec071020 2024-04-02T16:22:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.361+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f21ec06d2c0 con 0x7f21ec071020 2024-04-02T16:22:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.362+0000 7f21e8ff9640 1 -- 172.21.15.50:0/2832335649 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f21e005da50 con 0x7f21ec071020 2024-04-02T16:22:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.363+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f21d806c8f0 msgr2=0x7f21d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.363+0000 7f21f2782640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f21ec1f7270 tx=0x7f21dc007e40 comp rx=0 tx=0).stop 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.363+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21ec071020 msgr2=0x7f21ec0c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.363+0000 7f21f2782640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21ec071020 0x7f21ec0c90b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e870 tx=0x7f21e000ed30 comp rx=0 tx=0).stop 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 shutdown_connections 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21ec0d1440 0x7f21ec0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21ec09af80 0x7f21ec1f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 --2- 172.21.15.50:0/2832335649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21ec071020 0x7f21ec0c90b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 >> 172.21.15.50:0/2832335649 conn(0x7f21ec0b3ad0 msgr2=0x7f21ec15e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:56.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 shutdown_connections 2024-04-02T16:22:56.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.364+0000 7f21f2782640 1 -- 172.21.15.50:0/2832335649 wait complete. 2024-04-02T16:22:56.375 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-02T16:22:56.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:56.376 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-02T16:22:56.410 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:22:56.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:56.410 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-02T16:22:56.564 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:56.564 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:56.580 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:56.580 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:56.635 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:22:56.635 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:56.690 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:56.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.831+0000 7fc55b315640 1 Processor -- start 2024-04-02T16:22:56.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.831+0000 7fc55b315640 1 -- start start 2024-04-02T16:22:56.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc55b315640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc55415e570 0x7fc5540b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc55b315640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5540b0700 0x7fc5540ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc55b315640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc55406e8b0 0x7fc55406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc55b315640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fc554072a10 con 0x7fc55415e570 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc55b315640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc55415e950 con 0x7fc55406e8b0 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc55b315640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc55406c0b0 con 0x7fc5540b0700 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc55415e570 0x7fc5540b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc55415e570 0x7fc5540b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36782/0 (socket says 172.21.15.50:36782) 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 -- 172.21.15.50:0/2526970149 learned_addr learned my addr 172.21.15.50:0/2526970149 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5598b2640 1 --2- 172.21.15.50:0/2526970149 >> v2:172.21.15.188:3300/0 conn(0x7fc55406e8b0 0x7fc55406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5588b0640 1 --2- 172.21.15.50:0/2526970149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5540b0700 0x7fc5540ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 -- 172.21.15.50:0/2526970149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5540b0700 msgr2=0x7fc5540ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 --2- 172.21.15.50:0/2526970149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5540b0700 0x7fc5540ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 -- 172.21.15.50:0/2526970149 >> v2:172.21.15.188:3300/0 conn(0x7fc55406e8b0 msgr2=0x7fc55406ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:56.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 --2- 172.21.15.50:0/2526970149 >> v2:172.21.15.188:3300/0 conn(0x7fc55406e8b0 0x7fc55406ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5590b1640 1 -- 172.21.15.50:0/2526970149 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5540723c0 con 0x7fc55415e570 2024-04-02T16:22:56.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.832+0000 7fc5598b2640 1 --2- 172.21.15.50:0/2526970149 >> v2:172.21.15.188:3300/0 conn(0x7fc55406e8b0 0x7fc55406ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:56.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.833+0000 7fc5590b1640 1 --2- 172.21.15.50:0/2526970149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55415e570 0x7fc5540b49e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc5540b5dd0 tx=0x7fc55002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131890ab1721a7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:56.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.833+0000 7fc54bfff640 1 -- 172.21.15.50:0/2526970149 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55002e070 con 0x7fc55415e570 2024-04-02T16:22:56.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.833+0000 7fc54bfff640 1 -- 172.21.15.50:0/2526970149 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc550037070 con 0x7fc55415e570 2024-04-02T16:22:56.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.833+0000 7fc54bfff640 1 -- 172.21.15.50:0/2526970149 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550031a00 con 0x7fc55415e570 2024-04-02T16:22:56.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 -- 172.21.15.50:0/2526970149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55415e570 msgr2=0x7fc5540b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 --2- 172.21.15.50:0/2526970149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55415e570 0x7fc5540b49e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc5540b5dd0 tx=0x7fc55002b880 comp rx=0 tx=0).stop 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 -- 172.21.15.50:0/2526970149 shutdown_connections 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 --2- 172.21.15.50:0/2526970149 >> v2:172.21.15.188:3300/0 conn(0x7fc55406e8b0 0x7fc55406ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 --2- 172.21.15.50:0/2526970149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5540b0700 0x7fc5540ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 --2- 172.21.15.50:0/2526970149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc55415e570 0x7fc5540b49e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 -- 172.21.15.50:0/2526970149 >> 172.21.15.50:0/2526970149 conn(0x7fc5540b6540 msgr2=0x7fc5540b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 -- 172.21.15.50:0/2526970149 shutdown_connections 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.834+0000 7fc55b315640 1 -- 172.21.15.50:0/2526970149 wait complete. 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.835+0000 7fc55b315640 1 Processor -- start 2024-04-02T16:22:56.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.835+0000 7fc55b315640 1 -- start start 2024-04-02T16:22:56.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc55b315640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55406e8b0 0x7fc5541485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc55b315640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b0700 0x7fc5540ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc55b315640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc55415e570 0x7fc554159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc55b315640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc55406bcf0 con 0x7fc5540b0700 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc55b315640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc55406c390 con 0x7fc55406e8b0 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc55b315640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc55406be70 con 0x7fc55415e570 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b0700 0x7fc5540ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b0700 0x7fc5540ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36798/0 (socket says 172.21.15.50:36798) 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 -- 172.21.15.50:0/645786084 learned_addr learned my addr 172.21.15.50:0/645786084 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5590b1640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55406e8b0 0x7fc5541485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5598b2640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc55415e570 0x7fc554159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 -- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc55415e570 msgr2=0x7fc554159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc55415e570 0x7fc554159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 -- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55406e8b0 msgr2=0x7fc5541485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55406e8b0 0x7fc5541485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 -- 172.21.15.50:0/645786084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc554161970 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc5588b0640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b0700 0x7fc5540ce320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d7d0 tx=0x7fc54400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc549ffb640 1 -- 172.21.15.50:0/645786084 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544004020 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc549ffb640 1 -- 172.21.15.50:0/645786084 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5440041e0 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5540cd560 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.836+0000 7fc549ffb640 1 -- 172.21.15.50:0/645786084 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5440043d0 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.837+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5540cdb20 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.837+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5540756f0 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.838+0000 7fc549ffb640 1 -- 172.21.15.50:0/645786084 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc54401b020 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.838+0000 7fc549ffb640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.838+0000 7fc549ffb640 1 -- 172.21.15.50:0/645786084 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fc54408bcd0 con 0x7fc5540b0700 2024-04-02T16:22:56.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.838+0000 7fc5590b1640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:56.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.839+0000 7fc5590b1640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc53806c8f0 0x7fc53806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc550036000 tx=0x7fc55003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:56.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:56.843+0000 7fc549ffb640 1 -- 172.21.15.50:0/645786084 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc544059d70 con 0x7fc5540b0700 2024-04-02T16:22:57.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.027+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5540b1800 con 0x7fc5540b0700 2024-04-02T16:22:57.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.028+0000 7fc549ffb640 1 -- 172.21.15.50:0/645786084 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7fc544059670 con 0x7fc5540b0700 2024-04-02T16:22:57.029 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:57.029 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc53806c8f0 msgr2=0x7fc53806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc53806c8f0 0x7fc53806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc550036000 tx=0x7fc55003d040 comp rx=0 tx=0).stop 2024-04-02T16:22:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b0700 msgr2=0x7fc5540ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b0700 0x7fc5540ce320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d7d0 tx=0x7fc54400dc90 comp rx=0 tx=0).stop 2024-04-02T16:22:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 shutdown_connections 2024-04-02T16:22:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc55415e570 0x7fc554159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5540b0700 0x7fc5540ce320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 --2- 172.21.15.50:0/645786084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc55406e8b0 0x7fc5541485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 >> 172.21.15.50:0/645786084 conn(0x7fc5540b6540 msgr2=0x7fc5540694b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 shutdown_connections 2024-04-02T16:22:57.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.030+0000 7fc55b315640 1 -- 172.21.15.50:0/645786084 wait complete. 2024-04-02T16:22:57.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:57.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.146+0000 7f45a3b11640 1 Processor -- start 2024-04-02T16:22:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.146+0000 7f45a3b11640 1 -- start start 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a3b11640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c140fc0 0x7f459c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a3b11640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f459c0cef40 0x7f459c071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a3b11640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f459c141690 con 0x7f459c13e0b0 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a3b11640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f459c071360 con 0x7f459c0cef40 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a3b11640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f459c0cf3f0 con 0x7f459c140fc0 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a18ad640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f459c13e0b0 0x7f459c1408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47682/0 (socket says 172.21.15.50:47682) 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a18ad640 1 -- 172.21.15.50:0/1923460136 learned_addr learned my addr 172.21.15.50:0/1923460136 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a20ae640 1 --2- 172.21.15.50:0/1923460136 >> v2:172.21.15.188:3300/0 conn(0x7f459c0cef40 0x7f459c071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a10ac640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c140fc0 0x7f459c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808701942 0 0) 0x7f4598001110 con 0x7f459c13e0b0 2024-04-02T16:22:57.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4588003520 con 0x7f459c13e0b0 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2543555315 0 0) 0x7f4588003520 con 0x7f459c13e0b0 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c140fc0 msgr2=0x7f459c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c140fc0 0x7f459c0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 >> v2:172.21.15.188:3300/0 conn(0x7f459c0cef40 msgr2=0x7f459c071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 --2- 172.21.15.50:0/1923460136 >> v2:172.21.15.188:3300/0 conn(0x7f459c0cef40 0x7f459c071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c0cf670 con 0x7f459c13e0b0 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f4598001840 con 0x7f459c13e0b0 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 >> v1:172.21.15.50:6789/0 conn(0x7f459c13e0b0 legacy=0x7f459c1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45880039b0 0x7f4588003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45880043a0 0x7f4588004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4588004ef0 0x7f458800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f459c071360 con 0x7f45880043a0 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0cf3f0 con 0x7f4588004ef0 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f458800bf00 con 0x7f45880039b0 2024-04-02T16:22:57.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a20ae640 1 --2- 172.21.15.50:0/1923460136 >> v2:172.21.15.188:3300/0 conn(0x7f459c0cef40 0x7f459c071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.147+0000 7f45a10ac640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c140fc0 0x7f459c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a10ac640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45880043a0 0x7f4588004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a18ad640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45880039b0 0x7f4588003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a20ae640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4588004ef0 0x7f458800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a10ac640 1 -- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45880039b0 msgr2=0x7f4588003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a10ac640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45880039b0 0x7f4588003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a10ac640 1 -- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4588004ef0 msgr2=0x7f458800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a10ac640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4588004ef0 0x7f458800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.148+0000 7f45a10ac640 1 -- 172.21.15.50:0/1923460136 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4588003520 con 0x7f45880043a0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a10ac640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45880043a0 0x7f4588004880 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f459c144e70 tx=0x7f4588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffa6c4ae3ce1122 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c0097d0 con 0x7f45880043a0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458c004030 con 0x7f45880043a0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a08ab640 1 -- 172.21.15.50:0/1923460136 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c004400 con 0x7f45880043a0 2024-04-02T16:22:57.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a3b11640 1 -- 172.21.15.50:0/1923460136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45880043a0 msgr2=0x7f4588004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a3b11640 1 --2- 172.21.15.50:0/1923460136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45880043a0 0x7f4588004880 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f459c144e70 tx=0x7f4588005830 comp rx=0 tx=0).stop 2024-04-02T16:22:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a20ae640 1 -- 172.21.15.50:0/1923460136 reap_dead start 2024-04-02T16:22:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a3b11640 1 -- 172.21.15.50:0/1923460136 shutdown_connections 2024-04-02T16:22:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a3b11640 1 -- 172.21.15.50:0/1923460136 >> 172.21.15.50:0/1923460136 conn(0x7f459c06eb90 msgr2=0x7f459c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a3b11640 1 -- 172.21.15.50:0/1923460136 shutdown_connections 2024-04-02T16:22:57.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.149+0000 7f45a3b11640 1 -- 172.21.15.50:0/1923460136 wait complete. 2024-04-02T16:22:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.150+0000 7f45a3b11640 1 Processor -- start 2024-04-02T16:22:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a3b11640 1 -- start start 2024-04-02T16:22:57.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a3b11640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45880043a0 0x7f459c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a3b11640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c1e6c50 0x7f459c0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a3b11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c153110 0x7f459c153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a3b11640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0715b0 con 0x7f459c153110 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a3b11640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0cfc30 con 0x7f45880043a0 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a3b11640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f459c13e490 con 0x7f459c1e6c50 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a20ae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c153110 0x7f459c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a20ae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c153110 0x7f459c153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36824/0 (socket says 172.21.15.50:36824) 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a20ae640 1 -- 172.21.15.50:0/1573143182 learned_addr learned my addr 172.21.15.50:0/1573143182 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a10ac640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c1e6c50 0x7f459c0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.151+0000 7f45a18ad640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45880043a0 0x7f459c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a20ae640 1 -- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c1e6c50 msgr2=0x7f459c0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a20ae640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c1e6c50 0x7f459c0b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a20ae640 1 -- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45880043a0 msgr2=0x7f459c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a20ae640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45880043a0 0x7f459c15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a20ae640 1 -- 172.21.15.50:0/1573143182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c15ba40 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a20ae640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c153110 0x7f459c153590 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f459c0af5b0 tx=0x7f459400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45927fc640 1 -- 172.21.15.50:0/1573143182 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459400fdf0 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45927fc640 1 -- 172.21.15.50:0/1573143182 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4594003990 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c15bce0 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45927fc640 1 -- 172.21.15.50:0/1573143182 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594003d80 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.152+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c15c2a0 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.153+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c075760 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.153+0000 7f45927fc640 1 -- 172.21.15.50:0/1573143182 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f459401f020 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.154+0000 7f45927fc640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f458006c920 0x7f458006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.154+0000 7f45927fc640 1 -- 172.21.15.50:0/1573143182 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f459408c340 con 0x7f459c153110 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.154+0000 7f45a18ad640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f458006c920 0x7f458006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.154+0000 7f45a18ad640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f458006c920 0x7f458006ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45980012f0 tx=0x7f4598007ac0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.158+0000 7f45927fc640 1 -- 172.21.15.50:0/1573143182 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4594010040 con 0x7f459c153110 2024-04-02T16:22:57.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.341+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f459c0734d0 con 0x7f459c153110 2024-04-02T16:22:57.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.341+0000 7f45927fc640 1 -- 172.21.15.50:0/1573143182 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f4594059d00 con 0x7f459c153110 2024-04-02T16:22:57.343 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f458006c920 msgr2=0x7f458006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f458006c920 0x7f458006ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45980012f0 tx=0x7f4598007ac0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c153110 msgr2=0x7f459c153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c153110 0x7f459c153590 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f459c0af5b0 tx=0x7f459400f580 comp rx=0 tx=0).stop 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 shutdown_connections 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f458006c920 0x7f458006ede0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c153110 0x7f459c153590 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f459c1e6c50 0x7f459c0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 --2- 172.21.15.50:0/1573143182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45880043a0 0x7f459c15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 >> 172.21.15.50:0/1573143182 conn(0x7f459c06eb90 msgr2=0x7f459c0b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:57.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 shutdown_connections 2024-04-02T16:22:57.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.343+0000 7f45a3b11640 1 -- 172.21.15.50:0/1573143182 wait complete. 2024-04-02T16:22:57.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:57.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.461+0000 7f14a1532640 1 Processor -- start 2024-04-02T16:22:57.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.461+0000 7f14a1532640 1 -- start start 2024-04-02T16:22:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.461+0000 7f14a1532640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f149c15a630 0x7f149c071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.461+0000 7f14a1532640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c0b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.461+0000 7f14a1532640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149c0b47a0 con 0x7f149c0b3f10 2024-04-02T16:22:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.461+0000 7f14a1532640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f149c0b5030 con 0x7f149c15a630 2024-04-02T16:22:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.461+0000 7f14a1532640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f149c0b5310 con 0x7f149c15aa10 2024-04-02T16:22:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f149b7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f149c0b3f10 0x7f149c06d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47696/0 (socket says 172.21.15.50:47696) 2024-04-02T16:22:57.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f149b7fe640 1 -- 172.21.15.50:0/4012244480 learned_addr learned my addr 172.21.15.50:0/4012244480 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f1493fff640 1 --2- 172.21.15.50:0/4012244480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c0b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f149affd640 1 --2- 172.21.15.50:0/4012244480 >> v2:172.21.15.188:3300/0 conn(0x7f149c15a630 0x7f149c071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f149a7fc640 1 -- 172.21.15.50:0/4012244480 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143051864 0 0) 0x7f149c0b47a0 con 0x7f149c0b3f10 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f149a7fc640 1 -- 172.21.15.50:0/4012244480 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1488003520 con 0x7f149c0b3f10 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f1493fff640 1 -- 172.21.15.50:0/4012244480 >> v2:172.21.15.188:3300/0 conn(0x7f149c15a630 msgr2=0x7f149c071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f1493fff640 1 --2- 172.21.15.50:0/4012244480 >> v2:172.21.15.188:3300/0 conn(0x7f149c15a630 0x7f149c071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f1493fff640 1 -- 172.21.15.50:0/4012244480 >> v1:172.21.15.50:6789/0 conn(0x7f149c0b3f10 legacy=0x7f149c06d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.462+0000 7f1493fff640 1 -- 172.21.15.50:0/4012244480 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c0b31d0 con 0x7f149c15aa10 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.463+0000 7f1493fff640 1 --2- 172.21.15.50:0/4012244480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c0b39d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14840097a0 tx=0x7f148402b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be369b4bed439845 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.463+0000 7f149a7fc640 1 -- 172.21.15.50:0/4012244480 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484038070 con 0x7f149c15aa10 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.463+0000 7f149a7fc640 1 -- 172.21.15.50:0/4012244480 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1484031700 con 0x7f149c15aa10 2024-04-02T16:22:57.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.463+0000 7f149a7fc640 1 -- 172.21.15.50:0/4012244480 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484031ab0 con 0x7f149c15aa10 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.463+0000 7f14a1532640 1 -- 172.21.15.50:0/4012244480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 msgr2=0x7f149c0b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.463+0000 7f14a1532640 1 --2- 172.21.15.50:0/4012244480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c0b39d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14840097a0 tx=0x7f148402b7a0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.464+0000 7f14a1532640 1 -- 172.21.15.50:0/4012244480 shutdown_connections 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.464+0000 7f14a1532640 1 --2- 172.21.15.50:0/4012244480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c0b39d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.464+0000 7f14a1532640 1 --2- 172.21.15.50:0/4012244480 >> v2:172.21.15.188:3300/0 conn(0x7f149c15a630 0x7f149c071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.464+0000 7f14a1532640 1 -- 172.21.15.50:0/4012244480 >> 172.21.15.50:0/4012244480 conn(0x7f149c06eb90 msgr2=0x7f149c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.464+0000 7f14a1532640 1 -- 172.21.15.50:0/4012244480 shutdown_connections 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.464+0000 7f14a1532640 1 -- 172.21.15.50:0/4012244480 wait complete. 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 Processor -- start 2024-04-02T16:22:57.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 -- start start 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f149c15a630 0x7f149c152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f149c0c7620 0x7f149c14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f149c0b4c50 con 0x7f149c0c7620 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f149c0722b0 con 0x7f149c15a630 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.465+0000 7f14a1532640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f149c0b4dd0 con 0x7f149c15aa10 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f149c0c7620 0x7f149c14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f149c0c7620 0x7f149c14f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36840/0 (socket says 172.21.15.50:36840) 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 -- 172.21.15.50:0/4096431668 learned_addr learned my addr 172.21.15.50:0/4096431668 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149affd640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f149c15a630 0x7f149c152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f1493fff640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 -- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 msgr2=0x7f149c14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c14cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 -- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f149c15a630 msgr2=0x7f149c152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f149c15a630 0x7f149c152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 -- 172.21.15.50:0/4096431668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c0c6580 con 0x7f149c0c7620 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f149b7fe640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f149c0c7620 0x7f149c14f520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f149400cb70 tx=0x7f149400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f14937fe640 1 -- 172.21.15.50:0/4096431668 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494017070 con 0x7f149c0c7620 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f14937fe640 1 -- 172.21.15.50:0/4096431668 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149400ad40 con 0x7f149c0c7620 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149c0c6860 con 0x7f149c0c7620 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f14937fe640 1 -- 172.21.15.50:0/4096431668 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494012620 con 0x7f149c0c7620 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.466+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149c0c6e20 con 0x7f149c0c7620 2024-04-02T16:22:57.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.467+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149c075760 con 0x7f149c0c7620 2024-04-02T16:22:57.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.468+0000 7f14937fe640 1 -- 172.21.15.50:0/4096431668 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f149401a070 con 0x7f149c0c7620 2024-04-02T16:22:57.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.468+0000 7f14937fe640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f147c06c8f0 0x7f147c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.468+0000 7f14937fe640 1 -- 172.21.15.50:0/4096431668 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f149408bd30 con 0x7f149c0c7620 2024-04-02T16:22:57.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.468+0000 7f149affd640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f147c06c8f0 0x7f147c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.469+0000 7f149affd640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f147c06c8f0 0x7f147c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f148c00ac40 tx=0x7f148c00a2a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.473+0000 7f14937fe640 1 -- 172.21.15.50:0/4096431668 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f14940596f0 con 0x7f149c0c7620 2024-04-02T16:22:57.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.655+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f149c156ee0 con 0x7f149c0c7620 2024-04-02T16:22:57.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.656+0000 7f14937fe640 1 -- 172.21.15.50:0/4096431668 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f149405d9f0 con 0x7f149c0c7620 2024-04-02T16:22:57.658 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:57.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f147c06c8f0 msgr2=0x7f147c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f147c06c8f0 0x7f147c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f148c00ac40 tx=0x7f148c00a2a0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f149c0c7620 msgr2=0x7f149c14f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f149c0c7620 0x7f149c14f520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f149400cb70 tx=0x7f149400a430 comp rx=0 tx=0).stop 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 shutdown_connections 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f147c06c8f0 0x7f147c06edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f149c0c7620 0x7f149c14f520 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f149c15aa10 0x7f149c14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 --2- 172.21.15.50:0/4096431668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f149c15a630 0x7f149c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 >> 172.21.15.50:0/4096431668 conn(0x7f149c06eb90 msgr2=0x7f149c1561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 shutdown_connections 2024-04-02T16:22:57.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.658+0000 7f14a1532640 1 -- 172.21.15.50:0/4096431668 wait complete. 2024-04-02T16:22:57.670 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:57.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.773+0000 7f7b1c695640 1 Processor -- start 2024-04-02T16:22:57.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.773+0000 7f7b1c695640 1 -- start start 2024-04-02T16:22:57.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1c695640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b14161c80 0x7f7b140d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1c695640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7b140d13d0 0x7f7b140b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1c695640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b140b10e0 con 0x7f7b140d2e50 2024-04-02T16:22:57.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1c695640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f7b140b1380 con 0x7f7b140d13d0 2024-04-02T16:22:57.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1c695640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b140b1660 con 0x7f7b14161c80 2024-04-02T16:22:57.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1ac32640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7b140d13d0 0x7f7b140b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1ac32640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f7b140d13d0 0x7f7b140b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36854/0 (socket says 172.21.15.50:36854) 2024-04-02T16:22:57.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1ac32640 1 -- 172.21.15.50:0/1719631327 learned_addr learned my addr 172.21.15.50:0/1719631327 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:57.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.774+0000 7f7b1a431640 1 --2- 172.21.15.50:0/1719631327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b14161c80 0x7f7b140d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.775+0000 7f7b1ac32640 1 -- 172.21.15.50:0/1719631327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b14161c80 msgr2=0x7f7b140d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:57.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.775+0000 7f7b1ac32640 1 --2- 172.21.15.50:0/1719631327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b14161c80 0x7f7b140d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.775+0000 7f7b1ac32640 1 -- 172.21.15.50:0/1719631327 >> v1:172.21.15.188:6789/0 conn(0x7f7b140d2e50 legacy=0x7f7b140d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.775+0000 7f7b1ac32640 1 -- 172.21.15.50:0/1719631327 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b140d3340 con 0x7f7b140d13d0 2024-04-02T16:22:57.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.775+0000 7f7b1ac32640 1 --2- 172.21.15.50:0/1719631327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d13d0 0x7f7b140b1e60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009e50 tx=0x7f7b0402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130cbe48070e2e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1942f640 1 -- 172.21.15.50:0/1719631327 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0402ca70 con 0x7f7b140d13d0 2024-04-02T16:22:57.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1942f640 1 -- 172.21.15.50:0/1719631327 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0402cc10 con 0x7f7b140d13d0 2024-04-02T16:22:57.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1942f640 1 -- 172.21.15.50:0/1719631327 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040343f0 con 0x7f7b140d13d0 2024-04-02T16:22:57.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 -- 172.21.15.50:0/1719631327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d13d0 msgr2=0x7f7b140b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 --2- 172.21.15.50:0/1719631327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d13d0 0x7f7b140b1e60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009e50 tx=0x7f7b0402bed0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 -- 172.21.15.50:0/1719631327 shutdown_connections 2024-04-02T16:22:57.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 --2- 172.21.15.50:0/1719631327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d13d0 0x7f7b140b1e60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 --2- 172.21.15.50:0/1719631327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b14161c80 0x7f7b140d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 -- 172.21.15.50:0/1719631327 >> 172.21.15.50:0/1719631327 conn(0x7f7b1409d5e0 msgr2=0x7f7b140d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:57.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 -- 172.21.15.50:0/1719631327 shutdown_connections 2024-04-02T16:22:57.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.776+0000 7f7b1c695640 1 -- 172.21.15.50:0/1719631327 wait complete. 2024-04-02T16:22:57.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.777+0000 7f7b1c695640 1 Processor -- start 2024-04-02T16:22:57.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.777+0000 7f7b1c695640 1 -- start start 2024-04-02T16:22:57.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1c695640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b140d13d0 0x7f7b140b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1c695640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d2e50 0x7f7b140b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1c695640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b14161c80 0x7f7b140b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1c695640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7b140b2520 con 0x7f7b140d2e50 2024-04-02T16:22:57.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1c695640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b140b23a0 con 0x7f7b14161c80 2024-04-02T16:22:57.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1c695640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b140b26a0 con 0x7f7b140d13d0 2024-04-02T16:22:57.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d2e50 0x7f7b140b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d2e50 0x7f7b140b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36860/0 (socket says 172.21.15.50:36860) 2024-04-02T16:22:57.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 -- 172.21.15.50:0/774974474 learned_addr learned my addr 172.21.15.50:0/774974474 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:57.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1ac32640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b14161c80 0x7f7b140b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1a431640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b140d13d0 0x7f7b140b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 -- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b140d13d0 msgr2=0x7f7b140b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b140d13d0 0x7f7b140b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 -- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b14161c80 msgr2=0x7f7b140b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b14161c80 0x7f7b140b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b19c30640 1 -- 172.21.15.50:0/774974474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b140c8030 con 0x7f7b140d2e50 2024-04-02T16:22:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.778+0000 7f7b1a431640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b140d13d0 0x7f7b140b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.779+0000 7f7b19c30640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d2e50 0x7f7b140b4020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c440 tx=0x7f7b08001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.779+0000 7f7b12ffd640 1 -- 172.21.15.50:0/774974474 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080052d0 con 0x7f7b140d2e50 2024-04-02T16:22:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.779+0000 7f7b12ffd640 1 -- 172.21.15.50:0/774974474 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b08005470 con 0x7f7b140d2e50 2024-04-02T16:22:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.779+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b140c8310 con 0x7f7b140d2e50 2024-04-02T16:22:57.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.779+0000 7f7b12ffd640 1 -- 172.21.15.50:0/774974474 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08007260 con 0x7f7b140d2e50 2024-04-02T16:22:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.779+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b140c88d0 con 0x7f7b140d2e50 2024-04-02T16:22:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.780+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b14075760 con 0x7f7b140d2e50 2024-04-02T16:22:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.780+0000 7f7b12ffd640 1 -- 172.21.15.50:0/774974474 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b08007400 con 0x7f7b140d2e50 2024-04-02T16:22:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.781+0000 7f7b12ffd640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7b0006c8f0 0x7f7b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.781+0000 7f7b12ffd640 1 -- 172.21.15.50:0/774974474 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f7b0808b350 con 0x7f7b140d2e50 2024-04-02T16:22:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.781+0000 7f7b1a431640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7b0006c8f0 0x7f7b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:57.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.781+0000 7f7b1a431640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7b0006c8f0 0x7f7b0006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b141f7510 tx=0x7f7b0c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:57.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.785+0000 7f7b12ffd640 1 -- 172.21.15.50:0/774974474 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7b08058da0 con 0x7f7b140d2e50 2024-04-02T16:22:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.969+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7b1406c360 con 0x7f7b140d2e50 2024-04-02T16:22:57.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.969+0000 7f7b12ffd640 1 -- 172.21.15.50:0/774974474 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f7b0805d0a0 con 0x7f7b140d2e50 2024-04-02T16:22:57.971 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.971+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7b0006c8f0 msgr2=0x7f7b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.971+0000 7f7b1c695640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7b0006c8f0 0x7f7b0006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b141f7510 tx=0x7f7b0c008040 comp rx=0 tx=0).stop 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.971+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d2e50 msgr2=0x7f7b140b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.971+0000 7f7b1c695640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d2e50 0x7f7b140b4020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c440 tx=0x7f7b08001e70 comp rx=0 tx=0).stop 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.971+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 shutdown_connections 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.971+0000 7f7b1c695640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b14161c80 0x7f7b140b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.972+0000 7f7b1c695640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7b0006c8f0 0x7f7b0006edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.972+0000 7f7b1c695640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b140d2e50 0x7f7b140b4020 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.972+0000 7f7b1c695640 1 --2- 172.21.15.50:0/774974474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b140d13d0 0x7f7b140b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.972+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 >> 172.21.15.50:0/774974474 conn(0x7f7b1409d5e0 msgr2=0x7f7b140b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:57.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.972+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 shutdown_connections 2024-04-02T16:22:57.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:57.972+0000 7f7b1c695640 1 -- 172.21.15.50:0/774974474 wait complete. 2024-04-02T16:22:57.983 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:22:58.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.086+0000 7f14d5293640 1 Processor -- start 2024-04-02T16:22:58.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.086+0000 7f14d5293640 1 -- start start 2024-04-02T16:22:58.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14d5293640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14d015c340 0x7f14d00b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14d5293640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d006f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14d5293640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f14d00cd4c0 0x7f14d00cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14d5293640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f14d00d0710 con 0x7f14d00cd4c0 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14d5293640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f14d00cd030 con 0x7f14d015c340 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14d5293640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f14d00d0960 con 0x7f14d015c720 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14cf7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f14d00cd4c0 0x7f14d00cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14cf7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f14d00cd4c0 0x7f14d00cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36868/0 (socket says 172.21.15.50:36868) 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14cf7fe640 1 -- 172.21.15.50:0/3703391782 learned_addr learned my addr 172.21.15.50:0/3703391782 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14ce7fc640 1 --2- 172.21.15.50:0/3703391782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d006f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.087+0000 7f14ceffd640 1 --2- 172.21.15.50:0/3703391782 >> v2:172.21.15.188:3300/0 conn(0x7f14d015c340 0x7f14d00b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14cf7fe640 1 -- 172.21.15.50:0/3703391782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 msgr2=0x7f14d006f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14cf7fe640 1 --2- 172.21.15.50:0/3703391782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d006f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14cf7fe640 1 -- 172.21.15.50:0/3703391782 >> v2:172.21.15.188:3300/0 conn(0x7f14d015c340 msgr2=0x7f14d00b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:58.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14cf7fe640 1 --2- 172.21.15.50:0/3703391782 >> v2:172.21.15.188:3300/0 conn(0x7f14d015c340 0x7f14d00b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14cf7fe640 1 -- 172.21.15.50:0/3703391782 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d00d0c40 con 0x7f14d00cd4c0 2024-04-02T16:22:58.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14ceffd640 1 --2- 172.21.15.50:0/3703391782 >> v2:172.21.15.188:3300/0 conn(0x7f14d015c340 0x7f14d00b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:58.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14cf7fe640 1 --2- 172.21.15.50:0/3703391782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d00cd940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f14d0072040 tx=0x7f14c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f442bc6741b7391b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.088+0000 7f14cdffb640 1 -- 172.21.15.50:0/3703391782 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8035020 con 0x7f14d00cd4c0 2024-04-02T16:22:58.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14cdffb640 1 -- 172.21.15.50:0/3703391782 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c8031870 con 0x7f14d00cd4c0 2024-04-02T16:22:58.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14cdffb640 1 -- 172.21.15.50:0/3703391782 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8031c40 con 0x7f14d00cd4c0 2024-04-02T16:22:58.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 -- 172.21.15.50:0/3703391782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 msgr2=0x7f14d00cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 --2- 172.21.15.50:0/3703391782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d00cd940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f14d0072040 tx=0x7f14c802b910 comp rx=0 tx=0).stop 2024-04-02T16:22:58.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 -- 172.21.15.50:0/3703391782 shutdown_connections 2024-04-02T16:22:58.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 --2- 172.21.15.50:0/3703391782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d00cd940 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 --2- 172.21.15.50:0/3703391782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d006f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 --2- 172.21.15.50:0/3703391782 >> v2:172.21.15.188:3300/0 conn(0x7f14d015c340 0x7f14d00b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 -- 172.21.15.50:0/3703391782 >> 172.21.15.50:0/3703391782 conn(0x7f14d00b3c20 msgr2=0x7f14d00b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 -- 172.21.15.50:0/3703391782 shutdown_connections 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.089+0000 7f14d5293640 1 -- 172.21.15.50:0/3703391782 wait complete. 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.090+0000 7f14d5293640 1 Processor -- start 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.090+0000 7f14d5293640 1 -- start start 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14d5293640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d0150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14d5293640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14d015c340 0x7f14d015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14d5293640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14d5293640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14d00cde80 con 0x7f14d00cd4c0 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14d5293640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14d00ccc20 con 0x7f14d015c340 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14d5293640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f14d00ce000 con 0x7f14d015c720 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14ceffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d0150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14ceffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d0150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36878/0 (socket says 172.21.15.50:36878) 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14ceffd640 1 -- 172.21.15.50:0/2975837439 learned_addr learned my addr 172.21.15.50:0/2975837439 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:58.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14ce7fc640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14d015c340 0x7f14d015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14cf7fe640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.091+0000 7f14ceffd640 1 -- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 msgr2=0x7f14d015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14ceffd640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14ceffd640 1 -- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14d015c340 msgr2=0x7f14d015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14ceffd640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14d015c340 0x7f14d015d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14ceffd640 1 -- 172.21.15.50:0/2975837439 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d00cb0a0 con 0x7f14d00cd4c0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14ceffd640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d0150480 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f14c0007f20 tx=0x7f14c000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14b77fe640 1 -- 172.21.15.50:0/2975837439 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c001a070 con 0x7f14d00cd4c0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14b77fe640 1 -- 172.21.15.50:0/2975837439 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c0003b90 con 0x7f14d00cd4c0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d00cb380 con 0x7f14d00cd4c0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14b77fe640 1 -- 172.21.15.50:0/2975837439 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004030 con 0x7f14d00cd4c0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.092+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d00cb940 con 0x7f14d00cd4c0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.093+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d0075760 con 0x7f14d00cd4c0 2024-04-02T16:22:58.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.093+0000 7f14b77fe640 1 -- 172.21.15.50:0/2975837439 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14c001d070 con 0x7f14d00cd4c0 2024-04-02T16:22:58.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.094+0000 7f14b77fe640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f14bc06c8f0 0x7f14bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.094+0000 7f14b77fe640 1 -- 172.21.15.50:0/2975837439 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f14c008c1e0 con 0x7f14d00cd4c0 2024-04-02T16:22:58.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.094+0000 7f14ce7fc640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f14bc06c8f0 0x7f14bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.094+0000 7f14ce7fc640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f14bc06c8f0 0x7f14bc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14d00b15f0 tx=0x7f14c4009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.098+0000 7f14b77fe640 1 -- 172.21.15.50:0/2975837439 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f14c0010070 con 0x7f14d00cd4c0 2024-04-02T16:22:58.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.282+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f14d00b80b0 con 0x7f14d00cd4c0 2024-04-02T16:22:58.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.283+0000 7f14b77fe640 1 -- 172.21.15.50:0/2975837439 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f14c0059b00 con 0x7f14d00cd4c0 2024-04-02T16:22:58.284 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:58.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f14bc06c8f0 msgr2=0x7f14bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f14bc06c8f0 0x7f14bc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14d00b15f0 tx=0x7f14c4009190 comp rx=0 tx=0).stop 2024-04-02T16:22:58.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 msgr2=0x7f14d0150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d0150480 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f14c0007f20 tx=0x7f14c000d560 comp rx=0 tx=0).stop 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 shutdown_connections 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14d015c720 0x7f14d015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14d015c340 0x7f14d015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f14bc06c8f0 0x7f14bc06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 --2- 172.21.15.50:0/2975837439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d00cd4c0 0x7f14d0150480 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 >> 172.21.15.50:0/2975837439 conn(0x7f14d00b3c20 msgr2=0x7f14d00cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 shutdown_connections 2024-04-02T16:22:58.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.285+0000 7f14d5293640 1 -- 172.21.15.50:0/2975837439 wait complete. 2024-04-02T16:22:58.297 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:22:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.399+0000 7f9a406eb640 1 Processor -- start 2024-04-02T16:22:58.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.399+0000 7f9a406eb640 1 -- start start 2024-04-02T16:22:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.399+0000 7f9a406eb640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9a3813e120 0x7f9a380cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.399+0000 7f9a406eb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.399+0000 7f9a406eb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a380cf450 con 0x7f9a380cef40 2024-04-02T16:22:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.399+0000 7f9a406eb640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f9a380cf730 con 0x7f9a3813e120 2024-04-02T16:22:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.399+0000 7f9a406eb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a380cfa10 con 0x7f9a38141500 2024-04-02T16:22:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3e487640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9a3813e120 0x7f9a380cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3e487640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9a3813e120 0x7f9a380cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36886/0 (socket says 172.21.15.50:36886) 2024-04-02T16:22:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3e487640 1 -- 172.21.15.50:0/1510051757 learned_addr learned my addr 172.21.15.50:0/1510051757 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3dc86640 1 --2- 172.21.15.50:0/1510051757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3e487640 1 -- 172.21.15.50:0/1510051757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 msgr2=0x7f9a38140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3e487640 1 --2- 172.21.15.50:0/1510051757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3e487640 1 -- 172.21.15.50:0/1510051757 >> v1:172.21.15.188:6789/0 conn(0x7f9a380cef40 legacy=0x7f9a3815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.400+0000 7f9a3e487640 1 -- 172.21.15.50:0/1510051757 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a380cfcf0 con 0x7f9a3813e120 2024-04-02T16:22:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a3e487640 1 --2- 172.21.15.50:0/1510051757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a380cea00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009870 tx=0x7f9a3002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b78ded7aafa04ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a3d485640 1 -- 172.21.15.50:0/1510051757 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30034070 con 0x7f9a3813e120 2024-04-02T16:22:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a3d485640 1 -- 172.21.15.50:0/1510051757 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a30030840 con 0x7f9a3813e120 2024-04-02T16:22:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a3d485640 1 -- 172.21.15.50:0/1510051757 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30030c10 con 0x7f9a3813e120 2024-04-02T16:22:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a406eb640 1 -- 172.21.15.50:0/1510051757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 msgr2=0x7f9a380cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a406eb640 1 --2- 172.21.15.50:0/1510051757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a380cea00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009870 tx=0x7f9a3002b980 comp rx=0 tx=0).stop 2024-04-02T16:22:58.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a406eb640 1 -- 172.21.15.50:0/1510051757 shutdown_connections 2024-04-02T16:22:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a406eb640 1 --2- 172.21.15.50:0/1510051757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a406eb640 1 --2- 172.21.15.50:0/1510051757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a380cea00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a406eb640 1 -- 172.21.15.50:0/1510051757 >> 172.21.15.50:0/1510051757 conn(0x7f9a3806eb90 msgr2=0x7f9a3806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.401+0000 7f9a406eb640 1 -- 172.21.15.50:0/1510051757 shutdown_connections 2024-04-02T16:22:58.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.402+0000 7f9a406eb640 1 -- 172.21.15.50:0/1510051757 wait complete. 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 Processor -- start 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 -- start start 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cef40 0x7f9a38130090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a38134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38135500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38072930 con 0x7f9a3813e120 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3813e500 con 0x7f9a380cef40 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a406eb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a38072ab0 con 0x7f9a38141500 2024-04-02T16:22:58.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a3dc86640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a38134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a3dc86640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a38134fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36892/0 (socket says 172.21.15.50:36892) 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.403+0000 7f9a3dc86640 1 -- 172.21.15.50:0/144960468 learned_addr learned my addr 172.21.15.50:0/144960468 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3e487640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cef40 0x7f9a38130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3ec88640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3dc86640 1 -- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 msgr2=0x7f9a38135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3dc86640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3dc86640 1 -- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cef40 msgr2=0x7f9a38130090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3dc86640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cef40 0x7f9a38130090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3dc86640 1 -- 172.21.15.50:0/144960468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a38133620 con 0x7f9a3813e120 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a3dc86640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a38134fc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d5c0 tx=0x7f9a2800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a2effd640 1 -- 172.21.15.50:0/144960468 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280125f0 con 0x7f9a3813e120 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a2effd640 1 -- 172.21.15.50:0/144960468 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a28019070 con 0x7f9a3813e120 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a38133900 con 0x7f9a3813e120 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a2effd640 1 -- 172.21.15.50:0/144960468 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280129d0 con 0x7f9a3813e120 2024-04-02T16:22:58.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.404+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a38132870 con 0x7f9a3813e120 2024-04-02T16:22:58.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.405+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a381405c0 con 0x7f9a3813e120 2024-04-02T16:22:58.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.405+0000 7f9a2effd640 1 -- 172.21.15.50:0/144960468 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a28012b70 con 0x7f9a3813e120 2024-04-02T16:22:58.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.406+0000 7f9a2effd640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a1c06c8f0 0x7f9a1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.406+0000 7f9a2effd640 1 -- 172.21.15.50:0/144960468 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f9a2808c330 con 0x7f9a3813e120 2024-04-02T16:22:58.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.406+0000 7f9a3e487640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a1c06c8f0 0x7f9a1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.407+0000 7f9a3e487640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a1c06c8f0 0x7f9a1c06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9a30036000 tx=0x7f9a3003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.410+0000 7f9a2effd640 1 -- 172.21.15.50:0/144960468 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9a381405c0 con 0x7f9a3813e120 2024-04-02T16:22:58.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.606+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9a38075760 con 0x7f9a3813e120 2024-04-02T16:22:58.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.607+0000 7f9a2effd640 1 -- 172.21.15.50:0/144960468 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f9a2805a340 con 0x7f9a3813e120 2024-04-02T16:22:58.608 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a1c06c8f0 msgr2=0x7f9a1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a1c06c8f0 0x7f9a1c06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9a30036000 tx=0x7f9a3003b040 comp rx=0 tx=0).stop 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 msgr2=0x7f9a38134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a38134fc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d5c0 tx=0x7f9a2800da80 comp rx=0 tx=0).stop 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 shutdown_connections 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a38141500 0x7f9a38135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9a1c06c8f0 0x7f9a1c06edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a3813e120 0x7f9a38134fc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 --2- 172.21.15.50:0/144960468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a380cef40 0x7f9a38130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.608+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 >> 172.21.15.50:0/144960468 conn(0x7f9a3806eb90 msgr2=0x7f9a3813e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:58.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.609+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 shutdown_connections 2024-04-02T16:22:58.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.609+0000 7f9a406eb640 1 -- 172.21.15.50:0/144960468 wait complete. 2024-04-02T16:22:58.621 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-02T16:22:58.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:58.622 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-02T16:22:58.649 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T16:22:58.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:22:58.650 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-02T16:22:58.704 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:58.704 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:58.723 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:58.723 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:58.780 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T16:22:58.781 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:22:58.837 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:22:58.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.938+0000 7f3e122f2640 1 Processor -- start 2024-04-02T16:22:58.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e122f2640 1 -- start start 2024-04-02T16:22:58.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e122f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c0b4680 0x7f3e0c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e122f2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e0c09aa40 0x7f3e0c09aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e122f2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e0c0d0210 con 0x7f3e0c15e670 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e122f2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3e0c06da20 con 0x7f3e0c09aa40 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e122f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0c0d0b00 con 0x7f3e0c0b4680 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e0bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f3e0c15e670 0x7f3e0c0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47700/0 (socket says 172.21.15.50:47700) 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.939+0000 7f3e0bfff640 1 -- 172.21.15.50:0/1337540529 learned_addr learned my addr 172.21.15.50:0/1337540529 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 --2- 172.21.15.50:0/1337540529 >> v2:172.21.15.188:3300/0 conn(0x7f3e0c09aa40 0x7f3e0c09aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0b7fe640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c0b4680 0x7f3e0c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976693288 0 0) 0x7f3e0c0d0210 con 0x7f3e0c15e670 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3df4003520 con 0x7f3e0c15e670 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2132489652 0 0) 0x7f3df4003520 con 0x7f3e0c15e670 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c0b4680 msgr2=0x7f3e0c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c0b4680 0x7f3e0c06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 >> v2:172.21.15.188:3300/0 conn(0x7f3e0c09aa40 msgr2=0x7f3e0c09aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 --2- 172.21.15.50:0/1337540529 >> v2:172.21.15.188:3300/0 conn(0x7f3e0c09aa40 0x7f3e0c09aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0c06d740 con 0x7f3e0c15e670 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f3dfc001670 con 0x7f3e0c15e670 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 >> v1:172.21.15.50:6789/0 conn(0x7f3e0c15e670 legacy=0x7f3e0c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40039b0 0x7f3df4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df40043a0 0x7f3df4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3df4004ef0 0x7f3df400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c06da20 con 0x7f3df4004ef0 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c0d0b00 con 0x7f3df40043a0 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3df400bf00 con 0x7f3df40039b0 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 --2- 172.21.15.50:0/1337540529 >> v2:172.21.15.188:3300/0 conn(0x7f3e0c09aa40 0x7f3e0c09aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0b7fe640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c0b4680 0x7f3e0c06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3df4004ef0 0x7f3df400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0bfff640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40039b0 0x7f3df4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e0b7fe640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df40043a0 0x7f3df4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 -- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40039b0 msgr2=0x7f3df4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:58.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40039b0 0x7f3df4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 -- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df40043a0 msgr2=0x7f3df4004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df40043a0 0x7f3df4004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.940+0000 7f3e1088f640 1 -- 172.21.15.50:0/1337540529 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df4003520 con 0x7f3df4004ef0 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.941+0000 7f3e0b7fe640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3df40043a0 0x7f3df4004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.941+0000 7f3e1088f640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3df4004ef0 0x7f3df400b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400e380 tx=0x7f3df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23170aba6f26ef0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.941+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04009dc0 con 0x7f3df4004ef0 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.941+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0402cde0 con 0x7f3df4004ef0 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.941+0000 7f3e0affd640 1 -- 172.21.15.50:0/1337540529 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04003b90 con 0x7f3df4004ef0 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.942+0000 7f3e122f2640 1 -- 172.21.15.50:0/1337540529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3df4004ef0 msgr2=0x7f3df400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.942+0000 7f3e122f2640 1 --2- 172.21.15.50:0/1337540529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3df4004ef0 0x7f3df400b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400e380 tx=0x7f3df4005830 comp rx=0 tx=0).stop 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.942+0000 7f3e1088f640 1 -- 172.21.15.50:0/1337540529 reap_dead start 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.942+0000 7f3e122f2640 1 -- 172.21.15.50:0/1337540529 shutdown_connections 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.942+0000 7f3e122f2640 1 -- 172.21.15.50:0/1337540529 >> 172.21.15.50:0/1337540529 conn(0x7f3e0c0b3b90 msgr2=0x7f3e0c0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.942+0000 7f3e122f2640 1 -- 172.21.15.50:0/1337540529 shutdown_connections 2024-04-02T16:22:58.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.942+0000 7f3e122f2640 1 -- 172.21.15.50:0/1337540529 wait complete. 2024-04-02T16:22:58.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.943+0000 7f3e122f2640 1 Processor -- start 2024-04-02T16:22:58.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.943+0000 7f3e122f2640 1 -- start start 2024-04-02T16:22:58.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e122f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e0c1f40e0 0x7f3e0c1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e122f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c1f7d40 0x7f3e0c1f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e122f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e0c157710 0x7f3e0c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e122f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c0d0790 con 0x7f3e0c157710 2024-04-02T16:22:58.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e122f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c0d16a0 con 0x7f3e0c1f40e0 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e122f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0c06cff0 con 0x7f3e0c1f7d40 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e0c157710 0x7f3e0c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e0c157710 0x7f3e0c157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36916/0 (socket says 172.21.15.50:36916) 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 -- 172.21.15.50:0/160466277 learned_addr learned my addr 172.21.15.50:0/160466277 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e0b7fe640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c1f7d40 0x7f3e0c1f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e0bfff640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e0c1f40e0 0x7f3e0c1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 -- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c1f7d40 msgr2=0x7f3e0c1f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c1f7d40 0x7f3e0c1f81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 -- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e0c1f40e0 msgr2=0x7f3e0c1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e0c1f40e0 0x7f3e0c1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.944+0000 7f3e1088f640 1 -- 172.21.15.50:0/160466277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0c1e8820 con 0x7f3e0c157710 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.945+0000 7f3e1088f640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e0c157710 0x7f3e0c157b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e04036040 tx=0x7f3e04004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.945+0000 7f3e08ff9640 1 -- 172.21.15.50:0/160466277 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04009dc0 con 0x7f3e0c157710 2024-04-02T16:22:58.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.945+0000 7f3e08ff9640 1 -- 172.21.15.50:0/160466277 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e04045070 con 0x7f3e0c157710 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.945+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0c1e8ac0 con 0x7f3e0c157710 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.945+0000 7f3e08ff9640 1 -- 172.21.15.50:0/160466277 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04004560 con 0x7f3e0c157710 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.945+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0c1e9080 con 0x7f3e0c157710 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.946+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0c1e9740 con 0x7f3e0c157710 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.946+0000 7f3e08ff9640 1 -- 172.21.15.50:0/160466277 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e04005d10 con 0x7f3e0c157710 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.947+0000 7f3e08ff9640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3df806c8f0 0x7f3df806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.947+0000 7f3e08ff9640 1 -- 172.21.15.50:0/160466277 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f3e040ae570 con 0x7f3e0c157710 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.947+0000 7f3e0bfff640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3df806c8f0 0x7f3df806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:58.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.948+0000 7f3e0bfff640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3df806c8f0 0x7f3df806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:58.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:58.951+0000 7f3e08ff9640 1 -- 172.21.15.50:0/160466277 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3e0c1e9740 con 0x7f3e0c157710 2024-04-02T16:22:59.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.136+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e0c0756f0 con 0x7f3e0c157710 2024-04-02T16:22:59.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.137+0000 7f3e08ff9640 1 -- 172.21.15.50:0/160466277 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7f3e0403d330 con 0x7f3e0c157710 2024-04-02T16:22:59.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:22:59.139 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:22:59.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3df806c8f0 msgr2=0x7f3df806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3df806c8f0 0x7f3df806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc009040 comp rx=0 tx=0).stop 2024-04-02T16:22:59.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e0c157710 msgr2=0x7f3e0c157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e0c157710 0x7f3e0c157b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e04036040 tx=0x7f3e04004ab0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 shutdown_connections 2024-04-02T16:22:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3df806c8f0 0x7f3df806edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e0c157710 0x7f3e0c157b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e0c1f7d40 0x7f3e0c1f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 --2- 172.21.15.50:0/160466277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e0c1f40e0 0x7f3e0c1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.139+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 >> 172.21.15.50:0/160466277 conn(0x7f3e0c0b3b90 msgr2=0x7f3e0c0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.140+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 shutdown_connections 2024-04-02T16:22:59.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.140+0000 7f3e122f2640 1 -- 172.21.15.50:0/160466277 wait complete. 2024-04-02T16:22:59.152 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:22:59.153 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:22:59.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f640acd6640 1 Processor -- start 2024-04-02T16:22:59.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f640acd6640 1 -- start start 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f640acd6640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64040d1860 0x7f6404169360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f640acd6640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64040cf7b0 con 0x7f640415e5d0 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f640acd6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64040cfa90 con 0x7f64040d4170 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f640acd6640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f640415e9b0 con 0x7f64040d1860 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f6409273640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f64040d1860 0x7f6404169360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.258+0000 7f6408a72640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f640415e5d0 0x7f64040cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47714/0 (socket says 172.21.15.50:47714) 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.259+0000 7f6408a72640 1 -- 172.21.15.50:0/2045897477 learned_addr learned my addr 172.21.15.50:0/2045897477 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.259+0000 7f6409273640 1 -- 172.21.15.50:0/2045897477 >> v1:172.21.15.188:6789/0 conn(0x7f64040d4170 legacy=0x7f64040d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:59.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.259+0000 7f6409273640 1 -- 172.21.15.50:0/2045897477 >> v1:172.21.15.50:6789/0 conn(0x7f640415e5d0 legacy=0x7f64040cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.259+0000 7f64037fe640 1 -- 172.21.15.50:0/2045897477 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672799880 0 0) 0x7f64040cf7b0 con 0x7f640415e5d0 2024-04-02T16:22:59.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.259+0000 7f6409273640 1 -- 172.21.15.50:0/2045897477 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64040cfd70 con 0x7f64040d1860 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f6409273640 1 --2- 172.21.15.50:0/2045897477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64040d1860 0x7f6404169360 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009d90 tx=0x7f63fc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c6551f52e788df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f64037fe640 1 -- 172.21.15.50:0/2045897477 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc039070 con 0x7f64040d1860 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f64037fe640 1 -- 172.21.15.50:0/2045897477 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63fc02cc60 con 0x7f64040d1860 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f64037fe640 1 -- 172.21.15.50:0/2045897477 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc034660 con 0x7f64040d1860 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f640acd6640 1 -- 172.21.15.50:0/2045897477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64040d1860 msgr2=0x7f6404169360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f640acd6640 1 --2- 172.21.15.50:0/2045897477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64040d1860 0x7f6404169360 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009d90 tx=0x7f63fc02beb0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f640acd6640 1 -- 172.21.15.50:0/2045897477 shutdown_connections 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f640acd6640 1 --2- 172.21.15.50:0/2045897477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64040d1860 0x7f6404169360 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f640acd6640 1 -- 172.21.15.50:0/2045897477 >> 172.21.15.50:0/2045897477 conn(0x7f640408dce0 msgr2=0x7f640409d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:59.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f640acd6640 1 -- 172.21.15.50:0/2045897477 shutdown_connections 2024-04-02T16:22:59.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.260+0000 7f640acd6640 1 -- 172.21.15.50:0/2045897477 wait complete. 2024-04-02T16:22:59.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.261+0000 7f640acd6640 1 Processor -- start 2024-04-02T16:22:59.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.261+0000 7f640acd6640 1 -- start start 2024-04-02T16:22:59.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f640acd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64040d1860 0x7f640414b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f640acd6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64040d4170 0x7f64040b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f640acd6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f640415e5d0 0x7f640414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f640acd6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f64040cf3e0 con 0x7f640415e5d0 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f640acd6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64040d03c0 con 0x7f64040d4170 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f640acd6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f64040cf560 con 0x7f64040d1860 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f6409273640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f640415e5d0 0x7f640414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f6409273640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f640415e5d0 0x7f640414dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36930/0 (socket says 172.21.15.50:36930) 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.262+0000 7f6409273640 1 -- 172.21.15.50:0/4241134266 learned_addr learned my addr 172.21.15.50:0/4241134266 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6408a72640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64040d1860 0x7f640414b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6403fff640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64040d4170 0x7f64040b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6409273640 1 -- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64040d1860 msgr2=0x7f640414b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6409273640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64040d1860 0x7f640414b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6409273640 1 -- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64040d4170 msgr2=0x7f64040b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6409273640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64040d4170 0x7f64040b8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6409273640 1 -- 172.21.15.50:0/4241134266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64040b9250 con 0x7f640415e5d0 2024-04-02T16:22:59.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f6409273640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f640415e5d0 0x7f640414dff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009eb0 tx=0x7f63fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f64017fa640 1 -- 172.21.15.50:0/4241134266 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc044030 con 0x7f640415e5d0 2024-04-02T16:22:59.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f64017fa640 1 -- 172.21.15.50:0/4241134266 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63fc02ca60 con 0x7f640415e5d0 2024-04-02T16:22:59.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64040b9530 con 0x7f640415e5d0 2024-04-02T16:22:59.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f64017fa640 1 -- 172.21.15.50:0/4241134266 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc039040 con 0x7f640415e5d0 2024-04-02T16:22:59.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.263+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640414c000 con 0x7f640415e5d0 2024-04-02T16:22:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.264+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6404075760 con 0x7f640415e5d0 2024-04-02T16:22:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.264+0000 7f64017fa640 1 -- 172.21.15.50:0/4241134266 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63fc047070 con 0x7f640415e5d0 2024-04-02T16:22:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.265+0000 7f64017fa640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f63e806c8f0 0x7f63e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.265+0000 7f64017fa640 1 -- 172.21.15.50:0/4241134266 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f63fc0ae8a0 con 0x7f640415e5d0 2024-04-02T16:22:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.265+0000 7f6408a72640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f63e806c8f0 0x7f63e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.266+0000 7f6408a72640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f63e806c8f0 0x7f63e806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000f60 tx=0x7f63f8007c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.269+0000 7f64017fa640 1 -- 172.21.15.50:0/4241134266 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f63fc04e070 con 0x7f640415e5d0 2024-04-02T16:22:59.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.450+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f640414bc10 con 0x7f640415e5d0 2024-04-02T16:22:59.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.451+0000 7f64017fa640 1 -- 172.21.15.50:0/4241134266 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f63fc07c8b0 con 0x7f640415e5d0 2024-04-02T16:22:59.452 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:22:59.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.452+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f63e806c8f0 msgr2=0x7f63e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.452+0000 7f640acd6640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f63e806c8f0 0x7f63e806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000f60 tx=0x7f63f8007c00 comp rx=0 tx=0).stop 2024-04-02T16:22:59.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f640415e5d0 msgr2=0x7f640414dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f640415e5d0 0x7f640414dff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009eb0 tx=0x7f63fc004a40 comp rx=0 tx=0).stop 2024-04-02T16:22:59.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 shutdown_connections 2024-04-02T16:22:59.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f63e806c8f0 0x7f63e806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f640415e5d0 0x7f640414dff0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64040d4170 0x7f64040b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 --2- 172.21.15.50:0/4241134266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64040d1860 0x7f640414b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 >> 172.21.15.50:0/4241134266 conn(0x7f640408dce0 msgr2=0x7f640415eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:59.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 shutdown_connections 2024-04-02T16:22:59.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.453+0000 7f640acd6640 1 -- 172.21.15.50:0/4241134266 wait complete. 2024-04-02T16:22:59.464 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:22:59.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.568+0000 7fce415ba640 1 Processor -- start 2024-04-02T16:22:59.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.568+0000 7fce415ba640 1 -- start start 2024-04-02T16:22:59.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce415ba640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fce3c15d320 0x7fce3c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce415ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c0b2cb0 0x7fce3c0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce415ba640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce3c160520 con 0x7fce3c145be0 2024-04-02T16:22:59.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce415ba640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fce3c0cfd20 con 0x7fce3c15d320 2024-04-02T16:22:59.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce415ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c160800 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce3a7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fce3c15d320 0x7fce3c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce3a7fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fce3c15d320 0x7fce3c06f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36940/0 (socket says 172.21.15.50:36940) 2024-04-02T16:22:59.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.569+0000 7fce3a7fc640 1 -- 172.21.15.50:0/305860099 learned_addr learned my addr 172.21.15.50:0/305860099 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:59.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.570+0000 7fce3a7fc640 1 -- 172.21.15.50:0/305860099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c0b2cb0 msgr2=0x7fce3c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:59.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.570+0000 7fce3a7fc640 1 --2- 172.21.15.50:0/305860099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c0b2cb0 0x7fce3c0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.570+0000 7fce3a7fc640 1 -- 172.21.15.50:0/305860099 >> v1:172.21.15.188:6789/0 conn(0x7fce3c145be0 legacy=0x7fce3c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:59.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.570+0000 7fce3a7fc640 1 -- 172.21.15.50:0/305860099 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c160ae0 con 0x7fce3c15d320 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.570+0000 7fce3a7fc640 1 --2- 172.21.15.50:0/305860099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15d320 0x7fce3c06f920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e28e9a0276317ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.570+0000 7fce39ffb640 1 -- 172.21.15.50:0/305860099 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24035020 con 0x7fce3c15d320 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce39ffb640 1 -- 172.21.15.50:0/305860099 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce24031640 con 0x7fce3c15d320 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce39ffb640 1 -- 172.21.15.50:0/305860099 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2402d040 con 0x7fce3c15d320 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 -- 172.21.15.50:0/305860099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15d320 msgr2=0x7fce3c06f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 --2- 172.21.15.50:0/305860099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15d320 0x7fce3c06f920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce2402b8f0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 -- 172.21.15.50:0/305860099 shutdown_connections 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 --2- 172.21.15.50:0/305860099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c15d320 0x7fce3c06f920 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 --2- 172.21.15.50:0/305860099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c0b2cb0 0x7fce3c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 -- 172.21.15.50:0/305860099 >> 172.21.15.50:0/305860099 conn(0x7fce3c06e6c0 msgr2=0x7fce3c06eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:59.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 -- 172.21.15.50:0/305860099 shutdown_connections 2024-04-02T16:22:59.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.571+0000 7fce415ba640 1 -- 172.21.15.50:0/305860099 wait complete. 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.572+0000 7fce415ba640 1 Processor -- start 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.572+0000 7fce415ba640 1 -- start start 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce415ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c0b2cb0 0x7fce3c13d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce415ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3c145be0 0x7fce3c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce415ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c15d320 0x7fce3c0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce415ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0d01b0 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce415ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c06fe60 con 0x7fce3c145be0 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce415ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c0d0330 con 0x7fce3c15d320 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c0b2cb0 0x7fce3c13d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c0b2cb0 0x7fce3c13d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36948/0 (socket says 172.21.15.50:36948) 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 -- 172.21.15.50:0/253119242 learned_addr learned my addr 172.21.15.50:0/253119242 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 -- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c15d320 msgr2=0x7fce3c0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c15d320 0x7fce3c0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 -- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3c145be0 msgr2=0x7fce3c15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3c145be0 0x7fce3c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.573+0000 7fce3affd640 1 -- 172.21.15.50:0/253119242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c15b380 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.574+0000 7fce3affd640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c0b2cb0 0x7fce3c13d8b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c460 tx=0x7fce3400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.574+0000 7fce337fe640 1 -- 172.21.15.50:0/253119242 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34016020 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.574+0000 7fce337fe640 1 -- 172.21.15.50:0/253119242 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce34012870 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.574+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c15b660 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.574+0000 7fce337fe640 1 -- 172.21.15.50:0/253119242 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34012c60 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.574+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c15bc20 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.575+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3c075760 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.575+0000 7fce337fe640 1 -- 172.21.15.50:0/253119242 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce34019020 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.576+0000 7fce337fe640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fce1c06c8f0 0x7fce1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.576+0000 7fce337fe640 1 -- 172.21.15.50:0/253119242 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fce3408bd30 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.576+0000 7fce3a7fc640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fce1c06c8f0 0x7fce1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.577+0000 7fce3a7fc640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fce1c06c8f0 0x7fce1c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce24036000 tx=0x7fce24030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.580+0000 7fce337fe640 1 -- 172.21.15.50:0/253119242 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fce34020030 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.763+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fce3c072730 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.763+0000 7fce337fe640 1 -- 172.21.15.50:0/253119242 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fce34059ca0 con 0x7fce3c0b2cb0 2024-04-02T16:22:59.765 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fce1c06c8f0 msgr2=0x7fce1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fce1c06c8f0 0x7fce1c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce24036000 tx=0x7fce24030000 comp rx=0 tx=0).stop 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c0b2cb0 msgr2=0x7fce3c13d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c0b2cb0 0x7fce3c13d8b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c460 tx=0x7fce3400c920 comp rx=0 tx=0).stop 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 shutdown_connections 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce3c15d320 0x7fce3c0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce3c145be0 0x7fce3c15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fce1c06c8f0 0x7fce1c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 --2- 172.21.15.50:0/253119242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce3c0b2cb0 0x7fce3c13d8b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 >> 172.21.15.50:0/253119242 conn(0x7fce3c06e6c0 msgr2=0x7fce3c0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 shutdown_connections 2024-04-02T16:22:59.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.765+0000 7fce415ba640 1 -- 172.21.15.50:0/253119242 wait complete. 2024-04-02T16:22:59.777 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:22:59.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.881+0000 7f42d170e640 1 Processor -- start 2024-04-02T16:22:59.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.881+0000 7f42d170e640 1 -- start start 2024-04-02T16:22:59.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42d170e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f42cc0d4980 0x7f42cc162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42d170e640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f42cc0d0c80 0x7f42cc0d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42d170e640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42cc0d15d0 con 0x7f42cc161ca0 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42d170e640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f42cc1630f0 con 0x7f42cc0d0c80 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42d170e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f42cc0d5000 con 0x7f42cc0d4980 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42cb7fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f42cc0d0c80 0x7f42cc0d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42caffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f42cc161ca0 0x7f42cc0cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47718/0 (socket says 172.21.15.50:47718) 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42caffd640 1 -- 172.21.15.50:0/3472620991 learned_addr learned my addr 172.21.15.50:0/3472620991 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42ca7fc640 1 --2- 172.21.15.50:0/3472620991 >> v2:172.21.15.188:3300/0 conn(0x7f42cc0d4980 0x7f42cc162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42cb7fe640 1 -- 172.21.15.50:0/3472620991 >> v2:172.21.15.188:3300/0 conn(0x7f42cc0d4980 msgr2=0x7f42cc162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42cb7fe640 1 --2- 172.21.15.50:0/3472620991 >> v2:172.21.15.188:3300/0 conn(0x7f42cc0d4980 0x7f42cc162bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42c9ffb640 1 -- 172.21.15.50:0/3472620991 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964846738 0 0) 0x7f42c4001110 con 0x7f42cc161ca0 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42cb7fe640 1 -- 172.21.15.50:0/3472620991 >> v1:172.21.15.50:6789/0 conn(0x7f42cc161ca0 legacy=0x7f42cc0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42cb7fe640 1 -- 172.21.15.50:0/3472620991 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc0cf0d0 con 0x7f42cc0d0c80 2024-04-02T16:22:59.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.882+0000 7f42ca7fc640 1 --2- 172.21.15.50:0/3472620991 >> v2:172.21.15.188:3300/0 conn(0x7f42cc0d4980 0x7f42cc162bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.883+0000 7f42cb7fe640 1 --2- 172.21.15.50:0/3472620991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc0d0c80 0x7f42cc0d1090 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f42cc06bd50 tx=0x7f42bc02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad26201c8fafcd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.883+0000 7f42c9ffb640 1 -- 172.21.15.50:0/3472620991 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc039070 con 0x7f42cc0d0c80 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.883+0000 7f42c9ffb640 1 -- 172.21.15.50:0/3472620991 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42bc02cb60 con 0x7f42cc0d0c80 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.883+0000 7f42c9ffb640 1 -- 172.21.15.50:0/3472620991 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc034440 con 0x7f42cc0d0c80 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 -- 172.21.15.50:0/3472620991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc0d0c80 msgr2=0x7f42cc0d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 --2- 172.21.15.50:0/3472620991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc0d0c80 0x7f42cc0d1090 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f42cc06bd50 tx=0x7f42bc02bdb0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 -- 172.21.15.50:0/3472620991 shutdown_connections 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 --2- 172.21.15.50:0/3472620991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc0d0c80 0x7f42cc0d1090 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 --2- 172.21.15.50:0/3472620991 >> v2:172.21.15.188:3300/0 conn(0x7f42cc0d4980 0x7f42cc162bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 -- 172.21.15.50:0/3472620991 >> 172.21.15.50:0/3472620991 conn(0x7f42cc09d5c0 msgr2=0x7f42cc0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:22:59.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 -- 172.21.15.50:0/3472620991 shutdown_connections 2024-04-02T16:22:59.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.884+0000 7f42d170e640 1 -- 172.21.15.50:0/3472620991 wait complete. 2024-04-02T16:22:59.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.885+0000 7f42d170e640 1 Processor -- start 2024-04-02T16:22:59.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.885+0000 7f42d170e640 1 -- start start 2024-04-02T16:22:59.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42d170e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42cc0d0c80 0x7f42cc0ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42d170e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42cc0d4980 0x7f42cc0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42d170e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc161ca0 0x7f42cc0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42d170e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc0ced80 con 0x7f42cc161ca0 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42d170e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc0cf690 con 0x7f42cc0d0c80 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42d170e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f42cc0b70c0 con 0x7f42cc0d4980 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc161ca0 0x7f42cc0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc161ca0 0x7f42cc0c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36966/0 (socket says 172.21.15.50:36966) 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 -- 172.21.15.50:0/3170874126 learned_addr learned my addr 172.21.15.50:0/3170874126 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42caffd640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42cc0d0c80 0x7f42cc0ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42ca7fc640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42cc0d4980 0x7f42cc0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 -- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42cc0d4980 msgr2=0x7f42cc0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42cc0d4980 0x7f42cc0c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 -- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42cc0d0c80 msgr2=0x7f42cc0ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42cc0d0c80 0x7f42cc0ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:22:59.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 -- 172.21.15.50:0/3170874126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc0c85e0 con 0x7f42cc161ca0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42cb7fe640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc161ca0 0x7f42cc0c7f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009e80 tx=0x7f42bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42c37fe640 1 -- 172.21.15.50:0/3170874126 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc039050 con 0x7f42cc161ca0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42c37fe640 1 -- 172.21.15.50:0/3170874126 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42bc045070 con 0x7f42cc161ca0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42cc0c88c0 con 0x7f42cc161ca0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.886+0000 7f42c37fe640 1 -- 172.21.15.50:0/3170874126 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc03ea50 con 0x7f42cc161ca0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.887+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42cc0c8e80 con 0x7f42cc161ca0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.888+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42cc075760 con 0x7f42cc161ca0 2024-04-02T16:22:59.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.888+0000 7f42c37fe640 1 -- 172.21.15.50:0/3170874126 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42bc02cd10 con 0x7f42cc161ca0 2024-04-02T16:22:59.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.888+0000 7f42c37fe640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f42ac06c8f0 0x7f42ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:22:59.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.889+0000 7f42c37fe640 1 -- 172.21.15.50:0/3170874126 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f42bc0aeb40 con 0x7f42cc161ca0 2024-04-02T16:22:59.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.889+0000 7f42caffd640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f42ac06c8f0 0x7f42ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:22:59.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.889+0000 7f42caffd640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f42ac06c8f0 0x7f42ac06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000f60 tx=0x7f42c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:22:59.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:22:59.893+0000 7f42c37fe640 1 -- 172.21.15.50:0/3170874126 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f42bc07c620 con 0x7f42cc161ca0 2024-04-02T16:23:00.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.076+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f42cc0ba130 con 0x7f42cc161ca0 2024-04-02T16:23:00.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.076+0000 7f42c37fe640 1 -- 172.21.15.50:0/3170874126 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f42bc080920 con 0x7f42cc161ca0 2024-04-02T16:23:00.077 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:23:00.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f42ac06c8f0 msgr2=0x7f42ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f42ac06c8f0 0x7f42ac06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000f60 tx=0x7f42c4009040 comp rx=0 tx=0).stop 2024-04-02T16:23:00.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc161ca0 msgr2=0x7f42cc0c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc161ca0 0x7f42cc0c7f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009e80 tx=0x7f42bc004a40 comp rx=0 tx=0).stop 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 shutdown_connections 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f42ac06c8f0 0x7f42ac06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42cc161ca0 0x7f42cc0c7f40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42cc0d4980 0x7f42cc0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 --2- 172.21.15.50:0/3170874126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42cc0d0c80 0x7f42cc0ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 >> 172.21.15.50:0/3170874126 conn(0x7f42cc09d5c0 msgr2=0x7f42cc06d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 shutdown_connections 2024-04-02T16:23:00.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.078+0000 7f42d170e640 1 -- 172.21.15.50:0/3170874126 wait complete. 2024-04-02T16:23:00.090 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:23:00.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.193+0000 7f118792b640 1 Processor -- start 2024-04-02T16:23:00.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.193+0000 7f118792b640 1 -- start start 2024-04-02T16:23:00.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f118792b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1180162600 0x7f11800b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f118792b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11801629e0 0x7f11800cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f118792b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f118009b450 con 0x7f1180162f30 2024-04-02T16:23:00.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f118792b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f11800d0aa0 con 0x7f1180162600 2024-04-02T16:23:00.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f118792b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f118009b6f0 con 0x7f11801629e0 2024-04-02T16:23:00.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f1184ec6640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1180162f30 0x7f11800d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47728/0 (socket says 172.21.15.50:47728) 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f1184ec6640 1 -- 172.21.15.50:0/518311890 learned_addr learned my addr 172.21.15.50:0/518311890 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.194+0000 7f11856c7640 1 --2- 172.21.15.50:0/518311890 >> v2:172.21.15.188:3300/0 conn(0x7f1180162600 0x7f11800b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f1185ec8640 1 --2- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11801629e0 0x7f11800cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f117ffff640 1 -- 172.21.15.50:0/518311890 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2262457231 0 0) 0x7f118009b450 con 0x7f1180162f30 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f117ffff640 1 -- 172.21.15.50:0/518311890 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1168003520 con 0x7f1180162f30 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f11856c7640 1 -- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11801629e0 msgr2=0x7f11800cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f11856c7640 1 --2- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11801629e0 0x7f11800cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f11856c7640 1 -- 172.21.15.50:0/518311890 >> v1:172.21.15.50:6789/0 conn(0x7f1180162f30 legacy=0x7f11800d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f11856c7640 1 -- 172.21.15.50:0/518311890 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118009b9d0 con 0x7f1180162600 2024-04-02T16:23:00.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.195+0000 7f1185ec8640 1 --2- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11801629e0 0x7f11800cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:23:00.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f11856c7640 1 --2- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162600 0x7f11800b4740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f11780097d0 tx=0x7f117802b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83ddc443198b572c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f117ffff640 1 -- 172.21.15.50:0/518311890 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178035020 con 0x7f1180162600 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f117ffff640 1 -- 172.21.15.50:0/518311890 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1178031800 con 0x7f1180162600 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f117ffff640 1 -- 172.21.15.50:0/518311890 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178031bd0 con 0x7f1180162600 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f118792b640 1 -- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162600 msgr2=0x7f11800b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f118792b640 1 --2- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162600 0x7f11800b4740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f11780097d0 tx=0x7f117802b8a0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f118792b640 1 -- 172.21.15.50:0/518311890 shutdown_connections 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f118792b640 1 --2- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11801629e0 0x7f11800cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f118792b640 1 --2- 172.21.15.50:0/518311890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162600 0x7f11800b4740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f118792b640 1 -- 172.21.15.50:0/518311890 >> 172.21.15.50:0/518311890 conn(0x7f11800b3c20 msgr2=0x7f11800b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.196+0000 7f118792b640 1 -- 172.21.15.50:0/518311890 shutdown_connections 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.197+0000 7f118792b640 1 -- 172.21.15.50:0/518311890 wait complete. 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 Processor -- start 2024-04-02T16:23:00.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 -- start start 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1180162600 0x7f11800c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11801629e0 0x7f11801f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162f30 0x7f11801f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f11800d0350 con 0x7f11801629e0 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11800d1640 con 0x7f1180162f30 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f118792b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f11800d04d0 con 0x7f1180162600 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f1184ec6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11801629e0 0x7f11801f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f1184ec6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11801629e0 0x7f11801f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36976/0 (socket says 172.21.15.50:36976) 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.198+0000 7f1184ec6640 1 -- 172.21.15.50:0/2281684437 learned_addr learned my addr 172.21.15.50:0/2281684437 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:00.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1185ec8640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162f30 0x7f11801f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f11856c7640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1180162600 0x7f11800c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1184ec6640 1 -- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1180162600 msgr2=0x7f11800c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1184ec6640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1180162600 0x7f11800c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1184ec6640 1 -- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162f30 msgr2=0x7f11801f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1184ec6640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162f30 0x7f11801f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1184ec6640 1 -- 172.21.15.50:0/2281684437 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11800c9dc0 con 0x7f11801629e0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f11856c7640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1180162600 0x7f11800c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1185ec8640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162f30 0x7f11801f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f1184ec6640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11801629e0 0x7f11801f49c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1174001eb0 tx=0x7f117400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f117dffb640 1 -- 172.21.15.50:0/2281684437 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174017070 con 0x7f11801629e0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f117dffb640 1 -- 172.21.15.50:0/2281684437 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1174005d00 con 0x7f11801629e0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11800ca0a0 con 0x7f11801629e0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f117dffb640 1 -- 172.21.15.50:0/2281684437 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174007310 con 0x7f11801629e0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.199+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11800c9200 con 0x7f11801629e0 2024-04-02T16:23:00.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.200+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118006f5b0 con 0x7f11801629e0 2024-04-02T16:23:00.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.201+0000 7f117dffb640 1 -- 172.21.15.50:0/2281684437 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f117401a070 con 0x7f11801629e0 2024-04-02T16:23:00.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.201+0000 7f117dffb640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f116c06c8f0 0x7f116c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.201+0000 7f117dffb640 1 -- 172.21.15.50:0/2281684437 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f117408b3e0 con 0x7f11801629e0 2024-04-02T16:23:00.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.202+0000 7f11856c7640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f116c06c8f0 0x7f116c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.202+0000 7f11856c7640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f116c06c8f0 0x7f116c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1178036000 tx=0x7f1178008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.206+0000 7f117dffb640 1 -- 172.21.15.50:0/2281684437 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1174059350 con 0x7f11801629e0 2024-04-02T16:23:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.390+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f11801f4290 con 0x7f11801629e0 2024-04-02T16:23:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.391+0000 7f117dffb640 1 -- 172.21.15.50:0/2281684437 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f1174058c50 con 0x7f11801629e0 2024-04-02T16:23:00.392 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:23:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f116c06c8f0 msgr2=0x7f116c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f116c06c8f0 0x7f116c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1178036000 tx=0x7f1178008030 comp rx=0 tx=0).stop 2024-04-02T16:23:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11801629e0 msgr2=0x7f11801f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11801629e0 0x7f11801f49c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1174001eb0 tx=0x7f117400c820 comp rx=0 tx=0).stop 2024-04-02T16:23:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 shutdown_connections 2024-04-02T16:23:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1180162f30 0x7f11801f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f116c06c8f0 0x7f116c06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f11801629e0 0x7f11801f49c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 --2- 172.21.15.50:0/2281684437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1180162600 0x7f11800c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 >> 172.21.15.50:0/2281684437 conn(0x7f11800b3c20 msgr2=0x7f11800cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 shutdown_connections 2024-04-02T16:23:00.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.393+0000 7f118792b640 1 -- 172.21.15.50:0/2281684437 wait complete. 2024-04-02T16:23:00.405 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:23:00.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.506+0000 7f40bb8d3640 1 Processor -- start 2024-04-02T16:23:00.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.506+0000 7f40bb8d3640 1 -- start start 2024-04-02T16:23:00.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40bb8d3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b415e3d0 0x7f40b40d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40bb8d3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f40b415e7b0 0x7f40b40d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40bb8d3640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b40d2f30 con 0x7f40b40c7e30 2024-04-02T16:23:00.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40bb8d3640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f40b40cf000 con 0x7f40b415e7b0 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40bb8d3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40cf2e0 con 0x7f40b415e3d0 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b9e70640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f40b415e7b0 0x7f40b40d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b8e6e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f40b40c7e30 0x7f40b40d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47734/0 (socket says 172.21.15.50:47734) 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b9e70640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f40b415e7b0 0x7f40b40d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36978/0 (socket says 172.21.15.50:36978) 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b8e6e640 1 -- 172.21.15.50:0/3072640054 learned_addr learned my addr 172.21.15.50:0/3072640054 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b966f640 1 --2- 172.21.15.50:0/3072640054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b415e3d0 0x7f40b40d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b9e70640 1 -- 172.21.15.50:0/3072640054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b415e3d0 msgr2=0x7f40b40d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b9e70640 1 --2- 172.21.15.50:0/3072640054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b415e3d0 0x7f40b40d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b9e70640 1 -- 172.21.15.50:0/3072640054 >> v1:172.21.15.50:6789/0 conn(0x7f40b40c7e30 legacy=0x7f40b40d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40abfff640 1 -- 172.21.15.50:0/3072640054 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 478358645 0 0) 0x7f40b40d2f30 con 0x7f40b40c7e30 2024-04-02T16:23:00.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.507+0000 7f40b9e70640 1 -- 172.21.15.50:0/3072640054 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b40cf5c0 con 0x7f40b415e7b0 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.508+0000 7f40b9e70640 1 --2- 172.21.15.50:0/3072640054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b40d2880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009d90 tx=0x7f40ac02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0f72a5702230b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.508+0000 7f40abfff640 1 -- 172.21.15.50:0/3072640054 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac039070 con 0x7f40b415e7b0 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.508+0000 7f40abfff640 1 -- 172.21.15.50:0/3072640054 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ac02cc50 con 0x7f40b415e7b0 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.508+0000 7f40abfff640 1 -- 172.21.15.50:0/3072640054 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac034440 con 0x7f40b415e7b0 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3072640054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 msgr2=0x7f40b40d2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3072640054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b40d2880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009d90 tx=0x7f40ac02bea0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3072640054 shutdown_connections 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3072640054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b40d2880 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3072640054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b415e3d0 0x7f40b40d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3072640054 >> 172.21.15.50:0/3072640054 conn(0x7f40b408dc90 msgr2=0x7f40b409d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:00.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3072640054 shutdown_connections 2024-04-02T16:23:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.509+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3072640054 wait complete. 2024-04-02T16:23:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.510+0000 7f40bb8d3640 1 Processor -- start 2024-04-02T16:23:00.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.510+0000 7f40bb8d3640 1 -- start start 2024-04-02T16:23:00.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40bb8d3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b40c7e30 0x7f40b40ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40bb8d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b415e3d0 0x7f40b4159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40bb8d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b4160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40bb8d3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40d46f0 con 0x7f40b415e7b0 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40bb8d3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40d4570 con 0x7f40b415e3d0 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40bb8d3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40d4870 con 0x7f40b40c7e30 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b4160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b4160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36994/0 (socket says 172.21.15.50:36994) 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 -- 172.21.15.50:0/3733081656 learned_addr learned my addr 172.21.15.50:0/3733081656 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b8e6e640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b415e3d0 0x7f40b4159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b966f640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b40c7e30 0x7f40b40ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 -- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b40c7e30 msgr2=0x7f40b40ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b40c7e30 0x7f40b40ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 -- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b415e3d0 msgr2=0x7f40b4159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b415e3d0 0x7f40b4159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 -- 172.21.15.50:0/3733081656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b40cd320 con 0x7f40b415e7b0 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40b9e70640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b4160ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f40ac037040 tx=0x7f40ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40a9ffb640 1 -- 172.21.15.50:0/3733081656 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac039050 con 0x7f40b415e7b0 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40a9ffb640 1 -- 172.21.15.50:0/3733081656 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ac03a040 con 0x7f40b415e7b0 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b40cd600 con 0x7f40b415e7b0 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.511+0000 7f40a9ffb640 1 -- 172.21.15.50:0/3733081656 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac02ca50 con 0x7f40b415e7b0 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.512+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41e7f50 con 0x7f40b415e7b0 2024-04-02T16:23:00.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.513+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b4075760 con 0x7f40b415e7b0 2024-04-02T16:23:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.513+0000 7f40a9ffb640 1 -- 172.21.15.50:0/3733081656 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40ac047030 con 0x7f40b415e7b0 2024-04-02T16:23:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.514+0000 7f40a9ffb640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f409806c920 0x7f409806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.514+0000 7f40a9ffb640 1 -- 172.21.15.50:0/3733081656 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f40ac0adda0 con 0x7f40b415e7b0 2024-04-02T16:23:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.514+0000 7f40b966f640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f409806c920 0x7f409806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.514+0000 7f40b966f640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f409806c920 0x7f409806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006fd0 tx=0x7f40b0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.518+0000 7f40a9ffb640 1 -- 172.21.15.50:0/3733081656 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f40ac07bdb0 con 0x7f40b415e7b0 2024-04-02T16:23:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.714+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40b40cb170 con 0x7f40b415e7b0 2024-04-02T16:23:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.714+0000 7f40a9ffb640 1 -- 172.21.15.50:0/3733081656 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f40ac07b6b0 con 0x7f40b415e7b0 2024-04-02T16:23:00.716 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-04-02T16:23:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f409806c920 msgr2=0x7f409806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f409806c920 0x7f409806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006fd0 tx=0x7f40b0008040 comp rx=0 tx=0).stop 2024-04-02T16:23:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 msgr2=0x7f40b4160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b4160ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f40ac037040 tx=0x7f40ac004a40 comp rx=0 tx=0).stop 2024-04-02T16:23:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 shutdown_connections 2024-04-02T16:23:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f409806c920 0x7f409806ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40b415e7b0 0x7f40b4160ae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b415e3d0 0x7f40b4159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 --2- 172.21.15.50:0/3733081656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b40c7e30 0x7f40b40ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.716+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 >> 172.21.15.50:0/3733081656 conn(0x7f40b408dc90 msgr2=0x7f40b415c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.717+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 shutdown_connections 2024-04-02T16:23:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.717+0000 7f40bb8d3640 1 -- 172.21.15.50:0/3733081656 wait complete. 2024-04-02T16:23:00.730 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-02T16:23:00.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:00.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-02T16:23:00.765 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T16:23:00.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:00.766 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-02T16:23:00.824 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-02T16:23:00.824 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:23:00.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.925+0000 7fb4d38e4640 1 Processor -- start 2024-04-02T16:23:00.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.925+0000 7fb4d38e4640 1 -- start start 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d38e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc15e510 0x7fb4cc0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d38e4640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb4cc0d2800 0x7fb4cc0d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d38e4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4cc0ceda0 con 0x7fb4cc0d0e90 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d38e4640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc0d31c0 con 0x7fb4cc0d2800 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d38e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc0cf040 con 0x7fb4cc15e510 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d0e7f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb4cc0d2800 0x7fb4cc0d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d1e81640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb4cc0d0e90 0x7fb4cc0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47738/0 (socket says 172.21.15.50:47738) 2024-04-02T16:23:00.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d1e81640 1 -- 172.21.15.50:0/2308005142 learned_addr learned my addr 172.21.15.50:0/2308005142 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d1680640 1 --2- 172.21.15.50:0/2308005142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc15e510 0x7fb4cc0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4c3fff640 1 -- 172.21.15.50:0/2308005142 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860861885 0 0) 0x7fb4cc0ceda0 con 0x7fb4cc0d0e90 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4c3fff640 1 -- 172.21.15.50:0/2308005142 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4b8003520 con 0x7fb4cc0d0e90 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d0e7f640 1 -- 172.21.15.50:0/2308005142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc15e510 msgr2=0x7fb4cc0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d0e7f640 1 --2- 172.21.15.50:0/2308005142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc15e510 0x7fb4cc0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d0e7f640 1 -- 172.21.15.50:0/2308005142 >> v1:172.21.15.50:6789/0 conn(0x7fb4cc0d0e90 legacy=0x7fb4cc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d0e7f640 1 -- 172.21.15.50:0/2308005142 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0cf320 con 0x7fb4cc0d2800 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.926+0000 7fb4d1680640 1 --2- 172.21.15.50:0/2308005142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc15e510 0x7fb4cc0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.927+0000 7fb4d0e7f640 1 --2- 172.21.15.50:0/2308005142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc0d2800 0x7fb4cc0d2c80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0097a0 tx=0x7fb4bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9900c865a1837a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.927+0000 7fb4c3fff640 1 -- 172.21.15.50:0/2308005142 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0315a0 con 0x7fb4cc0d2800 2024-04-02T16:23:00.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.927+0000 7fb4c3fff640 1 -- 172.21.15.50:0/2308005142 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4bc037070 con 0x7fb4cc0d2800 2024-04-02T16:23:00.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.927+0000 7fb4c3fff640 1 -- 172.21.15.50:0/2308005142 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc031960 con 0x7fb4cc0d2800 2024-04-02T16:23:00.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 -- 172.21.15.50:0/2308005142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc0d2800 msgr2=0x7fb4cc0d2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/2308005142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc0d2800 0x7fb4cc0d2c80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0097a0 tx=0x7fb4bc02b840 comp rx=0 tx=0).stop 2024-04-02T16:23:00.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 -- 172.21.15.50:0/2308005142 shutdown_connections 2024-04-02T16:23:00.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/2308005142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc0d2800 0x7fb4cc0d2c80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/2308005142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc15e510 0x7fb4cc0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 -- 172.21.15.50:0/2308005142 >> 172.21.15.50:0/2308005142 conn(0x7fb4cc09d570 msgr2=0x7fb4cc0d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:00.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 -- 172.21.15.50:0/2308005142 shutdown_connections 2024-04-02T16:23:00.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.928+0000 7fb4d38e4640 1 -- 172.21.15.50:0/2308005142 wait complete. 2024-04-02T16:23:00.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.929+0000 7fb4d38e4640 1 Processor -- start 2024-04-02T16:23:00.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.929+0000 7fb4d38e4640 1 -- start start 2024-04-02T16:23:00.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d38e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc0d0e90 0x7fb4cc1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d38e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4cc0d2800 0x7fb4cc157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d38e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc15e510 0x7fb4cc159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d38e4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc0b3ad0 con 0x7fb4cc15e510 2024-04-02T16:23:00.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d38e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc15e8f0 con 0x7fb4cc0d2800 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d38e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc0b3c50 con 0x7fb4cc0d0e90 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc15e510 0x7fb4cc159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc15e510 0x7fb4cc159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37012/0 (socket says 172.21.15.50:37012) 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 -- 172.21.15.50:0/515005046 learned_addr learned my addr 172.21.15.50:0/515005046 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1680640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc0d0e90 0x7fb4cc1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d0e7f640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4cc0d2800 0x7fb4cc157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 -- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc0d0e90 msgr2=0x7fb4cc1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc0d0e90 0x7fb4cc1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 -- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4cc0d2800 msgr2=0x7fb4cc157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4cc0d2800 0x7fb4cc157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 -- 172.21.15.50:0/515005046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0c9690 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d1e81640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc15e510 0x7fb4cc159500 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400cb40 tx=0x7fb4c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4c1ffb640 1 -- 172.21.15.50:0/515005046 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4017070 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4c1ffb640 1 -- 172.21.15.50:0/515005046 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c400aca0 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc0c9970 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.930+0000 7fb4c1ffb640 1 -- 172.21.15.50:0/515005046 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4012620 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.931+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc0c9360 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.932+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4cc1609f0 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.932+0000 7fb4c1ffb640 1 -- 172.21.15.50:0/515005046 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4c401a070 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.932+0000 7fb4c1ffb640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb4b006c8f0 0x7fb4b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.933+0000 7fb4c1ffb640 1 -- 172.21.15.50:0/515005046 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fb4c408bcf0 con 0x7fb4cc15e510 2024-04-02T16:23:00.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.933+0000 7fb4d1680640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb4b006c8f0 0x7fb4b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:00.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.933+0000 7fb4d1680640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb4b006c8f0 0x7fb4b006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800aac0 tx=0x7fb4c8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:00.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:00.937+0000 7fb4c1ffb640 1 -- 172.21.15.50:0/515005046 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb4c4090070 con 0x7fb4cc15e510 2024-04-02T16:23:01.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.161+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4cc0756f0 con 0x7fb4cc15e510 2024-04-02T16:23:01.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.162+0000 7fb4c1ffb640 1 -- 172.21.15.50:0/515005046 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7fb4c4059740 con 0x7fb4cc15e510 2024-04-02T16:23:01.163 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:01.163 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:23:01.163 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:23:01.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb4b006c8f0 msgr2=0x7fb4b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb4b006c8f0 0x7fb4b006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800aac0 tx=0x7fb4c8008040 comp rx=0 tx=0).stop 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc15e510 msgr2=0x7fb4cc159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc15e510 0x7fb4cc159500 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400cb40 tx=0x7fb4c400a060 comp rx=0 tx=0).stop 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 shutdown_connections 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb4b006c8f0 0x7fb4b006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4cc15e510 0x7fb4cc159500 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4cc0d2800 0x7fb4cc157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 --2- 172.21.15.50:0/515005046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4cc0d0e90 0x7fb4cc1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 >> 172.21.15.50:0/515005046 conn(0x7fb4cc09d570 msgr2=0x7fb4cc15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 shutdown_connections 2024-04-02T16:23:01.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.164+0000 7fb4d38e4640 1 -- 172.21.15.50:0/515005046 wait complete. 2024-04-02T16:23:01.176 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:23:01.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.277+0000 7f0950e50640 1 Processor -- start 2024-04-02T16:23:01.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.277+0000 7f0950e50640 1 -- start start 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.277+0000 7f0950e50640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c06d800 0x7f094c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.277+0000 7f0950e50640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f094c06c760 con 0x7f094c15e370 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.277+0000 7f0950e50640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f094c06ca40 con 0x7f094c09af80 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.277+0000 7f0950e50640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f094c06d370 con 0x7f094c06d800 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f094a59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f094c15e370 0x7f094c0d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47748/0 (socket says 172.21.15.50:47748) 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f094a59c640 1 -- 172.21.15.50:0/1078678984 learned_addr learned my addr 172.21.15.50:0/1078678984 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0943fff640 1 --2- 172.21.15.50:0/1078678984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c06d800 0x7f094c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175400255 0 0) 0x7f094c06c760 con 0x7f094c15e370 2024-04-02T16:23:01.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0938003520 con 0x7f094c15e370 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 317611390 0 0) 0x7f094c06ca40 con 0x7f094c09af80 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f094c06c760 con 0x7f094c09af80 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0943fff640 1 -- 172.21.15.50:0/1078678984 >> v1:172.21.15.188:6789/0 conn(0x7f094c09af80 legacy=0x7f094c168350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0943fff640 1 -- 172.21.15.50:0/1078678984 >> v1:172.21.15.50:6789/0 conn(0x7f094c15e370 legacy=0x7f094c0d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 984726412 0 0) 0x7f0938003520 con 0x7f094c15e370 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.278+0000 7f0943fff640 1 -- 172.21.15.50:0/1078678984 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c06cd20 con 0x7f094c06d800 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.279+0000 7f0943fff640 1 --2- 172.21.15.50:0/1078678984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c06d800 0x7f094c09aa40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09340097d0 tx=0x7f093402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3e57a364e1537 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.279+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934037070 con 0x7f094c06d800 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.279+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0934030510 con 0x7f094c06d800 2024-04-02T16:23:01.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.279+0000 7f0949d9b640 1 -- 172.21.15.50:0/1078678984 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c001670 con 0x7f094c06d800 2024-04-02T16:23:01.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.280+0000 7f0950e50640 1 -- 172.21.15.50:0/1078678984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c06d800 msgr2=0x7f094c09aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.280+0000 7f0950e50640 1 --2- 172.21.15.50:0/1078678984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c06d800 0x7f094c09aa40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09340097d0 tx=0x7f093402b800 comp rx=0 tx=0).stop 2024-04-02T16:23:01.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.280+0000 7f0950e50640 1 -- 172.21.15.50:0/1078678984 shutdown_connections 2024-04-02T16:23:01.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.280+0000 7f0950e50640 1 --2- 172.21.15.50:0/1078678984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c06d800 0x7f094c09aa40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.280+0000 7f0950e50640 1 -- 172.21.15.50:0/1078678984 >> 172.21.15.50:0/1078678984 conn(0x7f094c0b3ba0 msgr2=0x7f094c0b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:01.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.280+0000 7f0950e50640 1 -- 172.21.15.50:0/1078678984 shutdown_connections 2024-04-02T16:23:01.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.280+0000 7f0950e50640 1 -- 172.21.15.50:0/1078678984 wait complete. 2024-04-02T16:23:01.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.281+0000 7f0950e50640 1 Processor -- start 2024-04-02T16:23:01.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.281+0000 7f0950e50640 1 -- start start 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f094c06d800 0x7f094c161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c09af80 0x7f094c150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f094c15e370 0x7f094c1f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f094c0d1640 con 0x7f094c15e370 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f094c0d14c0 con 0x7f094c06d800 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f094c15e750 con 0x7f094c09af80 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f094c15e370 0x7f094c1f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f094c15e370 0x7f094c1f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37018/0 (socket says 172.21.15.50:37018) 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 -- 172.21.15.50:0/3113600447 learned_addr learned my addr 172.21.15.50:0/3113600447 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0943fff640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c09af80 0x7f094c150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094a59c640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f094c06d800 0x7f094c161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 -- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c09af80 msgr2=0x7f094c150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c09af80 0x7f094c150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 -- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f094c06d800 msgr2=0x7f094c161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f094c06d800 0x7f094c161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 -- 172.21.15.50:0/3113600447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c15fb80 con 0x7f094c15e370 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094a59c640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f094c06d800 0x7f094c161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f094ad9d640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f094c15e370 0x7f094c1f40d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f094400c6e0 tx=0x7f094400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0942ffd640 1 -- 172.21.15.50:0/3113600447 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944003b70 con 0x7f094c15e370 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0942ffd640 1 -- 172.21.15.50:0/3113600447 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0944003d10 con 0x7f094c15e370 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094c15fe60 con 0x7f094c15e370 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0942ffd640 1 -- 172.21.15.50:0/3113600447 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440052d0 con 0x7f094c15e370 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.282+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094c160420 con 0x7f094c15e370 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.283+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094c0756f0 con 0x7f094c15e370 2024-04-02T16:23:01.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.284+0000 7f0942ffd640 1 -- 172.21.15.50:0/3113600447 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0944005470 con 0x7f094c15e370 2024-04-02T16:23:01.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.284+0000 7f0942ffd640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f092c06c8f0 0x7f092c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.284+0000 7f0942ffd640 1 -- 172.21.15.50:0/3113600447 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f094408bda0 con 0x7f094c15e370 2024-04-02T16:23:01.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.284+0000 7f094a59c640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f092c06c8f0 0x7f092c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.285+0000 7f094a59c640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f092c06c8f0 0x7f092c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f093c0011e0 tx=0x7f093c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.289+0000 7f0942ffd640 1 -- 172.21.15.50:0/3113600447 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0944059dc0 con 0x7f094c15e370 2024-04-02T16:23:01.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.474+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f094c0b8030 con 0x7f094c15e370 2024-04-02T16:23:01.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.476+0000 7f0942ffd640 1 -- 172.21.15.50:0/3113600447 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7f09440596c0 con 0x7f094c15e370 2024-04-02T16:23:01.477 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:01.477 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:23:01.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f092c06c8f0 msgr2=0x7f092c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f092c06c8f0 0x7f092c06edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f093c0011e0 tx=0x7f093c009040 comp rx=0 tx=0).stop 2024-04-02T16:23:01.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f094c15e370 msgr2=0x7f094c1f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f094c15e370 0x7f094c1f40d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f094400c6e0 tx=0x7f094400cba0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 shutdown_connections 2024-04-02T16:23:01.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f092c06c8f0 0x7f092c06edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f094c15e370 0x7f094c1f40d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f094c09af80 0x7f094c150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 --2- 172.21.15.50:0/3113600447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f094c06d800 0x7f094c161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 >> 172.21.15.50:0/3113600447 conn(0x7f094c0b3ba0 msgr2=0x7f094c0aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:01.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 shutdown_connections 2024-04-02T16:23:01.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.478+0000 7f0950e50640 1 -- 172.21.15.50:0/3113600447 wait complete. 2024-04-02T16:23:01.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:21:58.747655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:21:59.703020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T16:22:03.421200+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T16:23:01.490 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:23:01.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.593+0000 7f9d963af640 1 Processor -- start 2024-04-02T16:23:01.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.593+0000 7f9d963af640 1 -- start start 2024-04-02T16:23:01.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d963af640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9d9015e3d0 0x7f9d900d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d963af640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900c7e30 0x7f9d900d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d963af640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d900d2f30 con 0x7f9d9015e7b0 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d963af640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f9d900cf000 con 0x7f9d9015e3d0 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d963af640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d900cf2e0 con 0x7f9d900c7e30 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d9494c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f9d9015e7b0 0x7f9d900d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47758/0 (socket says 172.21.15.50:47758) 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d8ffff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9d9015e3d0 0x7f9d900d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.594+0000 7f9d9494c640 1 -- 172.21.15.50:0/430444518 learned_addr learned my addr 172.21.15.50:0/430444518 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8effd640 1 -- 172.21.15.50:0/430444518 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 210512512 0 0) 0x7f9d900d2f30 con 0x7f9d9015e7b0 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8f7fe640 1 --2- 172.21.15.50:0/430444518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900c7e30 0x7f9d900d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8effd640 1 -- 172.21.15.50:0/430444518 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d7c003520 con 0x7f9d9015e7b0 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8ffff640 1 -- 172.21.15.50:0/430444518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900c7e30 msgr2=0x7f9d900d4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8ffff640 1 --2- 172.21.15.50:0/430444518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900c7e30 0x7f9d900d4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8ffff640 1 -- 172.21.15.50:0/430444518 >> v1:172.21.15.50:6789/0 conn(0x7f9d9015e7b0 legacy=0x7f9d900d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8ffff640 1 -- 172.21.15.50:0/430444518 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d900cf5c0 con 0x7f9d9015e3d0 2024-04-02T16:23:01.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8f7fe640 1 --2- 172.21.15.50:0/430444518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900c7e30 0x7f9d900d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:01.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8ffff640 1 --2- 172.21.15.50:0/430444518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d9015e3d0 0x7f9d900d4030 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9d800097a0 tx=0x7f9d8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9879540629140553 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8effd640 1 -- 172.21.15.50:0/430444518 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80035020 con 0x7f9d9015e3d0 2024-04-02T16:23:01.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.595+0000 7f9d8effd640 1 -- 172.21.15.50:0/430444518 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d80031830 con 0x7f9d9015e3d0 2024-04-02T16:23:01.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d8effd640 1 -- 172.21.15.50:0/430444518 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80031be0 con 0x7f9d9015e3d0 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 -- 172.21.15.50:0/430444518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d9015e3d0 msgr2=0x7f9d900d4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 --2- 172.21.15.50:0/430444518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d9015e3d0 0x7f9d900d4030 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9d800097a0 tx=0x7f9d8002b8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 -- 172.21.15.50:0/430444518 shutdown_connections 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 --2- 172.21.15.50:0/430444518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900c7e30 0x7f9d900d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 --2- 172.21.15.50:0/430444518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d9015e3d0 0x7f9d900d4030 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 -- 172.21.15.50:0/430444518 >> 172.21.15.50:0/430444518 conn(0x7f9d9008dc90 msgr2=0x7f9d9009d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 -- 172.21.15.50:0/430444518 shutdown_connections 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.596+0000 7f9d963af640 1 -- 172.21.15.50:0/430444518 wait complete. 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.597+0000 7f9d963af640 1 Processor -- start 2024-04-02T16:23:01.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.597+0000 7f9d963af640 1 -- start start 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d963af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d900c7e30 0x7f9d901e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d963af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d9015e3d0 0x7f9d901e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d963af640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9015e7b0 0x7f9d901e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d963af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9d900d2a00 con 0x7f9d900c7e30 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d963af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9d900d2880 con 0x7f9d9015e3d0 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d963af640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d900d2b80 con 0x7f9d9015e7b0 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d900c7e30 0x7f9d901e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d900c7e30 0x7f9d901e8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37038/0 (socket says 172.21.15.50:37038) 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 -- 172.21.15.50:0/1954269996 learned_addr learned my addr 172.21.15.50:0/1954269996 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8f7fe640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d9015e3d0 0x7f9d901e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d9494c640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9015e7b0 0x7f9d901e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 -- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9015e7b0 msgr2=0x7f9d901e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9015e7b0 0x7f9d901e9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 -- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d9015e3d0 msgr2=0x7f9d901e94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d9015e3d0 0x7f9d901e94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d8ffff640 1 -- 172.21.15.50:0/1954269996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9015ecf0 con 0x7f9d900c7e30 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.598+0000 7f9d9494c640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9015e7b0 0x7f9d901e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.599+0000 7f9d8ffff640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d900c7e30 0x7f9d901e8fa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9d800098c0 tx=0x7f9d800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.599+0000 7f9d8cff9640 1 -- 172.21.15.50:0/1954269996 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80035040 con 0x7f9d900c7e30 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.599+0000 7f9d8cff9640 1 -- 172.21.15.50:0/1954269996 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d80003610 con 0x7f9d900c7e30 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.599+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d900b6fe0 con 0x7f9d900c7e30 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.599+0000 7f9d8cff9640 1 -- 172.21.15.50:0/1954269996 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d800039c0 con 0x7f9d900c7e30 2024-04-02T16:23:01.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.599+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9015ef90 con 0x7f9d900c7e30 2024-04-02T16:23:01.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.600+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d90075760 con 0x7f9d900c7e30 2024-04-02T16:23:01.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.600+0000 7f9d8cff9640 1 -- 172.21.15.50:0/1954269996 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d80038020 con 0x7f9d900c7e30 2024-04-02T16:23:01.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.600+0000 7f9d8cff9640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.601+0000 7f9d8cff9640 1 -- 172.21.15.50:0/1954269996 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f9d800ae280 con 0x7f9d900c7e30 2024-04-02T16:23:01.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.601+0000 7f9d8f7fe640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.601+0000 7f9d8f7fe640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d7006c8f0 0x7f9d7006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d900ca870 tx=0x7f9d78008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.605+0000 7f9d8cff9640 1 -- 172.21.15.50:0/1954269996 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9d8007c200 con 0x7f9d900c7e30 2024-04-02T16:23:01.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.787+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d900ce780 con 0x7f9d900c7e30 2024-04-02T16:23:01.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.787+0000 7f9d8cff9640 1 -- 172.21.15.50:0/1954269996 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9d8007bb00 con 0x7f9d900c7e30 2024-04-02T16:23:01.789 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d7006c8f0 msgr2=0x7f9d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d7006c8f0 0x7f9d7006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d900ca870 tx=0x7f9d78008040 comp rx=0 tx=0).stop 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d900c7e30 msgr2=0x7f9d901e8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d900c7e30 0x7f9d901e8fa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9d800098c0 tx=0x7f9d800046c0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 shutdown_connections 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9015e7b0 0x7f9d901e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d9015e3d0 0x7f9d901e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 --2- 172.21.15.50:0/1954269996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d900c7e30 0x7f9d901e8fa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 >> 172.21.15.50:0/1954269996 conn(0x7f9d9008dc90 msgr2=0x7f9d9006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 shutdown_connections 2024-04-02T16:23:01.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.789+0000 7f9d963af640 1 -- 172.21.15.50:0/1954269996 wait complete. 2024-04-02T16:23:01.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:23:01.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.905+0000 7fb10bc62640 1 Processor -- start 2024-04-02T16:23:01.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.906+0000 7fb10bc62640 1 -- start start 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.906+0000 7fb10bc62640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb104141030 0x7fb104166100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.906+0000 7fb10bc62640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1040cf450 con 0x7fb104141500 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.906+0000 7fb10bc62640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1040cf730 con 0x7fb10413e120 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.906+0000 7fb10bc62640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb1040cfa10 con 0x7fb104141030 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.907+0000 7fb10a1ff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb104141030 0x7fb104166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.907+0000 7fb1091fd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb104141500 0x7fb104140920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47772/0 (socket says 172.21.15.50:47772) 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.907+0000 7fb1091fd640 1 -- 172.21.15.50:0/918571426 learned_addr learned my addr 172.21.15.50:0/918571426 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:01.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.907+0000 7fb10a1ff640 1 -- 172.21.15.50:0/918571426 >> v1:172.21.15.188:6789/0 conn(0x7fb10413e120 legacy=0x7fb1040cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:01.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.907+0000 7fb10a1ff640 1 -- 172.21.15.50:0/918571426 >> v1:172.21.15.50:6789/0 conn(0x7fb104141500 legacy=0x7fb104140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.907+0000 7fb10a1ff640 1 -- 172.21.15.50:0/918571426 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1040cfcf0 con 0x7fb104141030 2024-04-02T16:23:01.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.907+0000 7fb1089fc640 1 -- 172.21.15.50:0/918571426 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604695255 0 0) 0x7fb1040cf450 con 0x7fb104141500 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb10a1ff640 1 --2- 172.21.15.50:0/918571426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141030 0x7fb104166100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8060990c8e5117 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb1089fc640 1 -- 172.21.15.50:0/918571426 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc039070 con 0x7fb104141030 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb1089fc640 1 -- 172.21.15.50:0/918571426 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc02cc60 con 0x7fb104141030 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb1089fc640 1 -- 172.21.15.50:0/918571426 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc034660 con 0x7fb104141030 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb10bc62640 1 -- 172.21.15.50:0/918571426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141030 msgr2=0x7fb104166100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb10bc62640 1 --2- 172.21.15.50:0/918571426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141030 0x7fb104166100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc02beb0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb10bc62640 1 -- 172.21.15.50:0/918571426 shutdown_connections 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb10bc62640 1 --2- 172.21.15.50:0/918571426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141030 0x7fb104166100 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.908+0000 7fb10bc62640 1 -- 172.21.15.50:0/918571426 >> 172.21.15.50:0/918571426 conn(0x7fb10406eb90 msgr2=0x7fb10406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:01.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.909+0000 7fb10bc62640 1 -- 172.21.15.50:0/918571426 shutdown_connections 2024-04-02T16:23:01.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.909+0000 7fb10bc62640 1 -- 172.21.15.50:0/918571426 wait complete. 2024-04-02T16:23:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 Processor -- start 2024-04-02T16:23:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 -- start start 2024-04-02T16:23:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10413e120 0x7fb10412e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb104141030 0x7fb10413f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141500 0x7fb10413fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1040cf110 con 0x7fb104141500 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb10413e500 con 0x7fb104141030 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.910+0000 7fb10bc62640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb104072930 con 0x7fb10413e120 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141500 0x7fb10413fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141500 0x7fb10413fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37046/0 (socket says 172.21.15.50:37046) 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 -- 172.21.15.50:0/3224303294 learned_addr learned my addr 172.21.15.50:0/3224303294 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb1099fe640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10413e120 0x7fb10412e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb1091fd640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb104141030 0x7fb10413f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 -- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10413e120 msgr2=0x7fb10412e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10413e120 0x7fb10412e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 -- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb104141030 msgr2=0x7fb10413f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb104141030 0x7fb10413f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 -- 172.21.15.50:0/3224303294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1040b38f0 con 0x7fb104141500 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb1099fe640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10413e120 0x7fb10412e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10a1ff640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141500 0x7fb10413fe50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb0fa7fc640 1 -- 172.21.15.50:0/3224303294 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc044020 con 0x7fb104141500 2024-04-02T16:23:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb0fa7fc640 1 -- 172.21.15.50:0/3224303294 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc03f470 con 0x7fb104141500 2024-04-02T16:23:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1040b3bd0 con 0x7fb104141500 2024-04-02T16:23:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb0fa7fc640 1 -- 172.21.15.50:0/3224303294 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc039040 con 0x7fb104141500 2024-04-02T16:23:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.911+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1040b4190 con 0x7fb104141500 2024-04-02T16:23:01.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.912+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb104075760 con 0x7fb104141500 2024-04-02T16:23:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.913+0000 7fb0fa7fc640 1 -- 172.21.15.50:0/3224303294 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0fc047020 con 0x7fb104141500 2024-04-02T16:23:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.913+0000 7fb0fa7fc640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb0e806c8f0 0x7fb0e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.913+0000 7fb0fa7fc640 1 -- 172.21.15.50:0/3224303294 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fb0fc0aeac0 con 0x7fb104141500 2024-04-02T16:23:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.913+0000 7fb1099fe640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb0e806c8f0 0x7fb0e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:01.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.914+0000 7fb1099fe640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb0e806c8f0 0x7fb0e806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb100001750 tx=0x7fb100009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:01.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:01.917+0000 7fb0fa7fc640 1 -- 172.21.15.50:0/3224303294 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb0fc04e030 con 0x7fb104141500 2024-04-02T16:23:02.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.100+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb1040734f0 con 0x7fb104141500 2024-04-02T16:23:02.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.100+0000 7fb0fa7fc640 1 -- 172.21.15.50:0/3224303294 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fb0fc07cb60 con 0x7fb104141500 2024-04-02T16:23:02.101 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb0e806c8f0 msgr2=0x7fb0e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb0e806c8f0 0x7fb0e806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb100001750 tx=0x7fb100009040 comp rx=0 tx=0).stop 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141500 msgr2=0x7fb10413fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141500 0x7fb10413fe50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc004a40 comp rx=0 tx=0).stop 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 shutdown_connections 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb0e806c8f0 0x7fb0e806edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb104141500 0x7fb10413fe50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb104141030 0x7fb10413f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 --2- 172.21.15.50:0/3224303294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10413e120 0x7fb10412e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.102+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 >> 172.21.15.50:0/3224303294 conn(0x7fb10406eb90 msgr2=0x7fb1041439f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.103+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 shutdown_connections 2024-04-02T16:23:02.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.103+0000 7fb10bc62640 1 -- 172.21.15.50:0/3224303294 wait complete. 2024-04-02T16:23:02.114 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:23:02.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.217+0000 7f4954a14640 1 Processor -- start 2024-04-02T16:23:02.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.217+0000 7f4954a14640 1 -- start start 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f4954a14640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f49500d0cc0 0x7f49500d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f4954a14640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49500b1e60 0x7f49500b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f4954a14640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49500b10e0 con 0x7f4950161c80 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f4954a14640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f49500d1660 con 0x7f49500d0cc0 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f4954a14640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f49500b1380 con 0x7f49500b1e60 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f494dd9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f49500d0cc0 0x7f49500d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f494e59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f4950161c80 0x7f49500d2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47774/0 (socket says 172.21.15.50:47774) 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.218+0000 7f494e59c640 1 -- 172.21.15.50:0/3573981050 learned_addr learned my addr 172.21.15.50:0/3573981050 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494ed9d640 1 --2- 172.21.15.50:0/3573981050 >> v2:172.21.15.188:3300/0 conn(0x7f49500b1e60 0x7f49500b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494dd9b640 1 -- 172.21.15.50:0/3573981050 >> v2:172.21.15.188:3300/0 conn(0x7f49500b1e60 msgr2=0x7f49500b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494dd9b640 1 --2- 172.21.15.50:0/3573981050 >> v2:172.21.15.188:3300/0 conn(0x7f49500b1e60 0x7f49500b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494dd9b640 1 -- 172.21.15.50:0/3573981050 >> v1:172.21.15.50:6789/0 conn(0x7f4950161c80 legacy=0x7f49500d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494dd9b640 1 -- 172.21.15.50:0/3573981050 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500b1660 con 0x7f49500d0cc0 2024-04-02T16:23:02.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494d59a640 1 -- 172.21.15.50:0/3573981050 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389892007 0 0) 0x7f49500b10e0 con 0x7f4950161c80 2024-04-02T16:23:02.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494ed9d640 1 --2- 172.21.15.50:0/3573981050 >> v2:172.21.15.188:3300/0 conn(0x7f49500b1e60 0x7f49500b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.219+0000 7f494dd9b640 1 --2- 172.21.15.50:0/3573981050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49500d0cc0 0x7f49500d1120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f495008e770 tx=0x7f493802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f350fe00c789530 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f494d59a640 1 -- 172.21.15.50:0/3573981050 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380315a0 con 0x7f49500d0cc0 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f494d59a640 1 -- 172.21.15.50:0/3573981050 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4938037070 con 0x7f49500d0cc0 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f494d59a640 1 -- 172.21.15.50:0/3573981050 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938031960 con 0x7f49500d0cc0 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 -- 172.21.15.50:0/3573981050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49500d0cc0 msgr2=0x7f49500d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 --2- 172.21.15.50:0/3573981050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49500d0cc0 0x7f49500d1120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f495008e770 tx=0x7f493802b840 comp rx=0 tx=0).stop 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 -- 172.21.15.50:0/3573981050 shutdown_connections 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 --2- 172.21.15.50:0/3573981050 >> v2:172.21.15.188:3300/0 conn(0x7f49500b1e60 0x7f49500b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 --2- 172.21.15.50:0/3573981050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49500d0cc0 0x7f49500d1120 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 -- 172.21.15.50:0/3573981050 >> 172.21.15.50:0/3573981050 conn(0x7f495009d5e0 msgr2=0x7f49500d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:02.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 -- 172.21.15.50:0/3573981050 shutdown_connections 2024-04-02T16:23:02.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.220+0000 7f4954a14640 1 -- 172.21.15.50:0/3573981050 wait complete. 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 Processor -- start 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 -- start start 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49500b1e60 0x7f49501f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49500d0cc0 0x7f49500ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4950161c80 0x7f495006c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49500d0180 con 0x7f4950161c80 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49500d3340 con 0x7f49500d0cc0 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f4954a14640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49500d0300 con 0x7f49500b1e60 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f494ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4950161c80 0x7f495006c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f494ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4950161c80 0x7f495006c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37062/0 (socket says 172.21.15.50:37062) 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.222+0000 7f494ed9d640 1 -- 172.21.15.50:0/1809412115 learned_addr learned my addr 172.21.15.50:0/1809412115 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494dd9b640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49500d0cc0 0x7f49500ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494e59c640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49500b1e60 0x7f49501f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494ed9d640 1 -- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49500b1e60 msgr2=0x7f49501f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494ed9d640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49500b1e60 0x7f49501f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494ed9d640 1 -- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49500d0cc0 msgr2=0x7f49500ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494ed9d640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49500d0cc0 0x7f49500ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494ed9d640 1 -- 172.21.15.50:0/1809412115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49501f4d70 con 0x7f4950161c80 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494dd9b640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49500d0cc0 0x7f49500ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f494ed9d640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4950161c80 0x7f495006c470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f494000cd50 tx=0x7f494000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f4946ffd640 1 -- 172.21.15.50:0/1809412115 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000f680 con 0x7f4950161c80 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f4946ffd640 1 -- 172.21.15.50:0/1809412115 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494000f840 con 0x7f4950161c80 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49500cab70 con 0x7f4950161c80 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f4946ffd640 1 -- 172.21.15.50:0/1809412115 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000fc30 con 0x7f4950161c80 2024-04-02T16:23:02.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.223+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49500cce00 con 0x7f4950161c80 2024-04-02T16:23:02.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.224+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4950075760 con 0x7f4950161c80 2024-04-02T16:23:02.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.225+0000 7f4946ffd640 1 -- 172.21.15.50:0/1809412115 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49400213f0 con 0x7f4950161c80 2024-04-02T16:23:02.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.225+0000 7f4946ffd640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f493006c920 0x7f493006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.225+0000 7f4946ffd640 1 -- 172.21.15.50:0/1809412115 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f494008cda0 con 0x7f4950161c80 2024-04-02T16:23:02.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.225+0000 7f494e59c640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f493006c920 0x7f493006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.226+0000 7f494e59c640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f493006c920 0x7f493006ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4948000f60 tx=0x7f4948007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.230+0000 7f4946ffd640 1 -- 172.21.15.50:0/1809412115 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f49500cab70 con 0x7f4950161c80 2024-04-02T16:23:02.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.412+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f49500cad50 con 0x7f4950161c80 2024-04-02T16:23:02.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.413+0000 7f4946ffd640 1 -- 172.21.15.50:0/1809412115 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f49501f4d70 con 0x7f4950161c80 2024-04-02T16:23:02.414 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.414+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f493006c920 msgr2=0x7f493006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.414+0000 7f4954a14640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f493006c920 0x7f493006ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4948000f60 tx=0x7f4948007b50 comp rx=0 tx=0).stop 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.414+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4950161c80 msgr2=0x7f495006c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.414+0000 7f4954a14640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4950161c80 0x7f495006c470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f494000cd50 tx=0x7f494000a650 comp rx=0 tx=0).stop 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 shutdown_connections 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f493006c920 0x7f493006ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4950161c80 0x7f495006c470 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49500d0cc0 0x7f49500ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 --2- 172.21.15.50:0/1809412115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49500b1e60 0x7f49501f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 >> 172.21.15.50:0/1809412115 conn(0x7f495009d5e0 msgr2=0x7f495006d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:02.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 shutdown_connections 2024-04-02T16:23:02.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.415+0000 7f4954a14640 1 -- 172.21.15.50:0/1809412115 wait complete. 2024-04-02T16:23:02.426 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T16:23:02.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:23:02.471 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:23:02.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.531+0000 7fb409a55640 1 Processor -- start 2024-04-02T16:23:02.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.531+0000 7fb409a55640 1 -- start start 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.531+0000 7fb409a55640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb40415e700 0x7fb4040d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb409a55640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40415eae0 con 0x7fb4040d2fe0 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb409a55640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4040d2610 con 0x7fb4040cf910 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb409a55640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb4040b4310 con 0x7fb40415e700 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb402ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb40415e700 0x7fb4040d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb4037fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb4040d2fe0 0x7fb4040b47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47780/0 (socket says 172.21.15.50:47780) 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb4037fe640 1 -- 172.21.15.50:0/77002931 learned_addr learned my addr 172.21.15.50:0/77002931 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb402ffd640 1 -- 172.21.15.50:0/77002931 >> v1:172.21.15.188:6789/0 conn(0x7fb4040cf910 legacy=0x7fb4040d28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb402ffd640 1 -- 172.21.15.50:0/77002931 >> v1:172.21.15.50:6789/0 conn(0x7fb4040d2fe0 legacy=0x7fb4040b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.532+0000 7fb402ffd640 1 -- 172.21.15.50:0/77002931 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040cfe60 con 0x7fb40415e700 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.533+0000 7fb402ffd640 1 --2- 172.21.15.50:0/77002931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40415e700 0x7fb4040d20d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40097a0 tx=0x7fb3f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2957bb77d84b04bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.533+0000 7fb401ffb640 1 -- 172.21.15.50:0/77002931 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4034070 con 0x7fb40415e700 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.533+0000 7fb401ffb640 1 -- 172.21.15.50:0/77002931 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f4030610 con 0x7fb40415e700 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.533+0000 7fb401ffb640 1 -- 172.21.15.50:0/77002931 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40309e0 con 0x7fb40415e700 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.533+0000 7fb409a55640 1 -- 172.21.15.50:0/77002931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40415e700 msgr2=0x7fb4040d20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.533+0000 7fb409a55640 1 --2- 172.21.15.50:0/77002931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40415e700 0x7fb4040d20d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40097a0 tx=0x7fb3f402b960 comp rx=0 tx=0).stop 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.534+0000 7fb409a55640 1 -- 172.21.15.50:0/77002931 shutdown_connections 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.534+0000 7fb409a55640 1 --2- 172.21.15.50:0/77002931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb40415e700 0x7fb4040d20d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.534+0000 7fb409a55640 1 -- 172.21.15.50:0/77002931 >> 172.21.15.50:0/77002931 conn(0x7fb40409d5c0 msgr2=0x7fb4040d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.534+0000 7fb409a55640 1 -- 172.21.15.50:0/77002931 shutdown_connections 2024-04-02T16:23:02.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.534+0000 7fb409a55640 1 -- 172.21.15.50:0/77002931 wait complete. 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.535+0000 7fb409a55640 1 Processor -- start 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.535+0000 7fb409a55640 1 -- start start 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.535+0000 7fb409a55640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4040cf910 0x7fb4041579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.535+0000 7fb409a55640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4040d2fe0 0x7fb4040ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.535+0000 7fb409a55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb40415e700 0x7fb4040b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb409a55640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040b3fa0 con 0x7fb4040d2fe0 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb409a55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040b3cc0 con 0x7fb40415e700 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb409a55640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040b4eb0 con 0x7fb4040cf910 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4040d2fe0 0x7fb4040ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4040d2fe0 0x7fb4040ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37078/0 (socket says 172.21.15.50:37078) 2024-04-02T16:23:02.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 -- 172.21.15.50:0/338601694 learned_addr learned my addr 172.21.15.50:0/338601694 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb402ffd640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4040cf910 0x7fb4041579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4037fe640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb40415e700 0x7fb4040b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 -- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4040cf910 msgr2=0x7fb4041579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4040cf910 0x7fb4041579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 -- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb40415e700 msgr2=0x7fb4040b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb40415e700 0x7fb4040b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 -- 172.21.15.50:0/338601694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040cc7e0 con 0x7fb4040d2fe0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb4027fc640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4040d2fe0 0x7fb4040ca540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c400 tx=0x7fb3f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb3ebfff640 1 -- 172.21.15.50:0/338601694 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80052d0 con 0x7fb4040d2fe0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb3ebfff640 1 -- 172.21.15.50:0/338601694 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f8005490 con 0x7fb4040d2fe0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4040ccac0 con 0x7fb4040d2fe0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb3ebfff640 1 -- 172.21.15.50:0/338601694 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8007260 con 0x7fb4040d2fe0 2024-04-02T16:23:02.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.536+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4040cd080 con 0x7fb4040d2fe0 2024-04-02T16:23:02.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.538+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb404075760 con 0x7fb4040d2fe0 2024-04-02T16:23:02.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.538+0000 7fb3ebfff640 1 -- 172.21.15.50:0/338601694 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3f8007400 con 0x7fb4040d2fe0 2024-04-02T16:23:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.538+0000 7fb3ebfff640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3f006c8f0 0x7fb3f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.538+0000 7fb3ebfff640 1 -- 172.21.15.50:0/338601694 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fb3f808b390 con 0x7fb4040d2fe0 2024-04-02T16:23:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.538+0000 7fb402ffd640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3f006c8f0 0x7fb3f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.539+0000 7fb402ffd640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3f006c8f0 0x7fb3f006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4036000 tx=0x7fb3f403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.543+0000 7fb3ebfff640 1 -- 172.21.15.50:0/338601694 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb3f8058de0 con 0x7fb4040d2fe0 2024-04-02T16:23:02.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.727+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb4040ce690 con 0x7fb4040d2fe0 2024-04-02T16:23:02.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.727+0000 7fb3ebfff640 1 -- 172.21.15.50:0/338601694 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb3f805d0e0 con 0x7fb4040d2fe0 2024-04-02T16:23:02.728 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T16:23:02.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3f006c8f0 msgr2=0x7fb3f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3f006c8f0 0x7fb3f006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4036000 tx=0x7fb3f403b040 comp rx=0 tx=0).stop 2024-04-02T16:23:02.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4040d2fe0 msgr2=0x7fb4040ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4040d2fe0 0x7fb4040ca540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c400 tx=0x7fb3f800c8c0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 shutdown_connections 2024-04-02T16:23:02.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb40415e700 0x7fb4040b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb3f006c8f0 0x7fb3f006edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4040d2fe0 0x7fb4040ca540 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 --2- 172.21.15.50:0/338601694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4040cf910 0x7fb4041579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 >> 172.21.15.50:0/338601694 conn(0x7fb40409d5c0 msgr2=0x7fb4040d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:02.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 shutdown_connections 2024-04-02T16:23:02.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.729+0000 7fb409a55640 1 -- 172.21.15.50:0/338601694 wait complete. 2024-04-02T16:23:02.741 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:23:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.882+0000 7f461b2d9640 1 Processor -- start 2024-04-02T16:23:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.882+0000 7f461b2d9640 1 -- start start 2024-04-02T16:23:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.882+0000 7f461b2d9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f461415e670 0x7f46140b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.882+0000 7f461b2d9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f461409aa40 0x7f461409aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.882+0000 7f461b2d9640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46140d0210 con 0x7f461406cff0 2024-04-02T16:23:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.882+0000 7f461b2d9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f461406da20 con 0x7f461409aa40 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f461b2d9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f46140d0b00 con 0x7f461415e670 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4619876640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f461409aa40 0x7f461409aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4618874640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f461406cff0 0x7f46140d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47788/0 (socket says 172.21.15.50:47788) 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4618874640 1 -- 172.21.15.50:0/2827891046 learned_addr learned my addr 172.21.15.50:0/2827891046 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4619075640 1 --2- 172.21.15.50:0/2827891046 >> v2:172.21.15.188:3300/0 conn(0x7f461415e670 0x7f46140b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4619876640 1 -- 172.21.15.50:0/2827891046 >> v2:172.21.15.188:3300/0 conn(0x7f461415e670 msgr2=0x7f46140b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4619876640 1 --2- 172.21.15.50:0/2827891046 >> v2:172.21.15.188:3300/0 conn(0x7f461415e670 0x7f46140b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4619876640 1 -- 172.21.15.50:0/2827891046 >> v1:172.21.15.50:6789/0 conn(0x7f461406cff0 legacy=0x7f46140d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f4619876640 1 -- 172.21.15.50:0/2827891046 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461406d480 con 0x7f461409aa40 2024-04-02T16:23:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.883+0000 7f460bfff640 1 -- 172.21.15.50:0/2827891046 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 386395173 0 0) 0x7f46140d0210 con 0x7f461406cff0 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f4619876640 1 --2- 172.21.15.50:0/2827891046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461409aed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46140727a0 tx=0x7f460c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca08d2a6ea90095 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f460bfff640 1 -- 172.21.15.50:0/2827891046 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c034070 con 0x7f461409aa40 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f460bfff640 1 -- 172.21.15.50:0/2827891046 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c030610 con 0x7f461409aa40 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f460bfff640 1 -- 172.21.15.50:0/2827891046 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0309e0 con 0x7f461409aa40 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f461b2d9640 1 -- 172.21.15.50:0/2827891046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 msgr2=0x7f461409aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f461b2d9640 1 --2- 172.21.15.50:0/2827891046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461409aed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46140727a0 tx=0x7f460c02b960 comp rx=0 tx=0).stop 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f461b2d9640 1 -- 172.21.15.50:0/2827891046 shutdown_connections 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f461b2d9640 1 --2- 172.21.15.50:0/2827891046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461409aed0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f461b2d9640 1 --2- 172.21.15.50:0/2827891046 >> v2:172.21.15.188:3300/0 conn(0x7f461415e670 0x7f46140b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f461b2d9640 1 -- 172.21.15.50:0/2827891046 >> 172.21.15.50:0/2827891046 conn(0x7f46140b3b90 msgr2=0x7f46140b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.884+0000 7f461b2d9640 1 -- 172.21.15.50:0/2827891046 shutdown_connections 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.885+0000 7f461b2d9640 1 -- 172.21.15.50:0/2827891046 wait complete. 2024-04-02T16:23:02.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 Processor -- start 2024-04-02T16:23:02.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 -- start start 2024-04-02T16:23:02.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f461406cff0 0x7f4614147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f461415e670 0x7f46140b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f46140d04b0 con 0x7f461409aa40 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46140d16a0 con 0x7f461406cff0 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f461b2d9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f46140d0790 con 0x7f461415e670 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f4618874640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f4618874640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461415ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37108/0 (socket says 172.21.15.50:37108) 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.886+0000 7f4618874640 1 -- 172.21.15.50:0/1408840064 learned_addr learned my addr 172.21.15.50:0/1408840064 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4619075640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f461406cff0 0x7f4614147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4618874640 1 -- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f461415e670 msgr2=0x7f46140b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4619876640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f461415e670 0x7f46140b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4618874640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f461415e670 0x7f46140b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4618874640 1 -- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f461406cff0 msgr2=0x7f4614147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4618874640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f461406cff0 0x7f4614147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4618874640 1 -- 172.21.15.50:0/1408840064 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140b4cc0 con 0x7f461409aa40 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4619075640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f461406cff0 0x7f4614147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4618874640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461415ed00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f460400c4d0 tx=0x7f460400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4609ffb640 1 -- 172.21.15.50:0/1408840064 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604018070 con 0x7f461409aa40 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4609ffb640 1 -- 172.21.15.50:0/1408840064 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604007500 con 0x7f461409aa40 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4614147a40 con 0x7f461409aa40 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f4609ffb640 1 -- 172.21.15.50:0/1408840064 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604004a20 con 0x7f461409aa40 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.887+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46140b5240 con 0x7f461409aa40 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.888+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46140756f0 con 0x7f461409aa40 2024-04-02T16:23:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.889+0000 7f4609ffb640 1 -- 172.21.15.50:0/1408840064 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f460401b070 con 0x7f461409aa40 2024-04-02T16:23:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.889+0000 7f4609ffb640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f45f806c8f0 0x7f45f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.889+0000 7f4609ffb640 1 -- 172.21.15.50:0/1408840064 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f460408b330 con 0x7f461409aa40 2024-04-02T16:23:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.889+0000 7f4619075640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f45f806c8f0 0x7f45f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.890+0000 7f4619075640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f45f806c8f0 0x7f45f806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4610007eb0 tx=0x7f4610007400 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:02.894+0000 7f4609ffb640 1 -- 172.21.15.50:0/1408840064 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f46040592b0 con 0x7f461409aa40 2024-04-02T16:23:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.117+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f461415dcb0 con 0x7f461409aa40 2024-04-02T16:23:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.117+0000 7f4609ffb640 1 -- 172.21.15.50:0/1408840064 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f4604058bb0 con 0x7f461409aa40 2024-04-02T16:23:03.119 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:03.119 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:23:03.119 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.119+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f45f806c8f0 msgr2=0x7f45f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.119+0000 7f461b2d9640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f45f806c8f0 0x7f45f806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4610007eb0 tx=0x7f4610007400 comp rx=0 tx=0).stop 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.119+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 msgr2=0x7f461415ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461415ed00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f460400c4d0 tx=0x7f460400c990 comp rx=0 tx=0).stop 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 shutdown_connections 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f461415e670 0x7f46140b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f45f806c8f0 0x7f45f806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f461409aa40 0x7f461415ed00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 --2- 172.21.15.50:0/1408840064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f461406cff0 0x7f4614147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 >> 172.21.15.50:0/1408840064 conn(0x7f46140b3b90 msgr2=0x7f4614070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 shutdown_connections 2024-04-02T16:23:03.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.120+0000 7f461b2d9640 1 -- 172.21.15.50:0/1408840064 wait complete. 2024-04-02T16:23:03.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:23:03.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.232+0000 7ff4850d3640 1 Processor -- start 2024-04-02T16:23:03.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.232+0000 7ff4850d3640 1 -- start start 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff4850d3640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff48015e670 0x7ff4800b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff4850d3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff48006cff0 0x7ff4800d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff4850d3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4800d0210 con 0x7ff48009aa40 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff4850d3640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7ff48006da20 con 0x7ff48015e670 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff4850d3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff4800d0b00 con 0x7ff48006cff0 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff47ed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff48015e670 0x7ff4800b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff47ed9d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7ff48015e670 0x7ff4800b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37122/0 (socket says 172.21.15.50:37122) 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff47ed9d640 1 -- 172.21.15.50:0/2511182331 learned_addr learned my addr 172.21.15.50:0/2511182331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:03.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.233+0000 7ff477fff640 1 --2- 172.21.15.50:0/2511182331 >> v2:172.21.15.188:3300/0 conn(0x7ff48006cff0 0x7ff4800d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.234+0000 7ff47ed9d640 1 -- 172.21.15.50:0/2511182331 >> v2:172.21.15.188:3300/0 conn(0x7ff48006cff0 msgr2=0x7ff4800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.234+0000 7ff47ed9d640 1 --2- 172.21.15.50:0/2511182331 >> v2:172.21.15.188:3300/0 conn(0x7ff48006cff0 0x7ff4800d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.234+0000 7ff47ed9d640 1 -- 172.21.15.50:0/2511182331 >> v1:172.21.15.188:6789/0 conn(0x7ff48009aa40 legacy=0x7ff48006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.234+0000 7ff47ed9d640 1 -- 172.21.15.50:0/2511182331 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48006d480 con 0x7ff48015e670 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.234+0000 7ff47ed9d640 1 --2- 172.21.15.50:0/2511182331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b4680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff480072c50 tx=0x7ff47002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9023f06a0e6c1c09 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff47e59c640 1 -- 172.21.15.50:0/2511182331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470034070 con 0x7ff48015e670 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff47e59c640 1 -- 172.21.15.50:0/2511182331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff470030860 con 0x7ff48015e670 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff47e59c640 1 -- 172.21.15.50:0/2511182331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470030c30 con 0x7ff48015e670 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 -- 172.21.15.50:0/2511182331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 msgr2=0x7ff4800b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 --2- 172.21.15.50:0/2511182331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b4680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff480072c50 tx=0x7ff47002b9a0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 -- 172.21.15.50:0/2511182331 shutdown_connections 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 --2- 172.21.15.50:0/2511182331 >> v2:172.21.15.188:3300/0 conn(0x7ff48006cff0 0x7ff4800d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 --2- 172.21.15.50:0/2511182331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b4680 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 -- 172.21.15.50:0/2511182331 >> 172.21.15.50:0/2511182331 conn(0x7ff4800b3b90 msgr2=0x7ff4800b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:03.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 -- 172.21.15.50:0/2511182331 shutdown_connections 2024-04-02T16:23:03.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.235+0000 7ff4850d3640 1 -- 172.21.15.50:0/2511182331 wait complete. 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.236+0000 7ff4850d3640 1 Processor -- start 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.236+0000 7ff4850d3640 1 -- start start 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff4850d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48006cff0 0x7ff4801f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff4850d3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48009aa40 0x7ff48006fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff4850d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff4850d3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4800d0790 con 0x7ff48015e670 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff4850d3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4800d04b0 con 0x7ff48006cff0 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff4850d3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff4800d14d0 con 0x7ff48009aa40 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff47f59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff47f59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37124/0 (socket says 172.21.15.50:37124) 2024-04-02T16:23:03.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff47f59e640 1 -- 172.21.15.50:0/1431040569 learned_addr learned my addr 172.21.15.50:0/1431040569 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff47ed9d640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48006cff0 0x7ff4801f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.237+0000 7ff477fff640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48009aa40 0x7ff48006fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff47f59e640 1 -- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48009aa40 msgr2=0x7ff48006fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff47f59e640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48009aa40 0x7ff48006fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff47f59e640 1 -- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48006cff0 msgr2=0x7ff4801f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff47f59e640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48006cff0 0x7ff4801f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff47f59e640 1 -- 172.21.15.50:0/1431040569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4801f3df0 con 0x7ff48015e670 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff47ed9d640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48006cff0 0x7ff4801f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff47f59e640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b6e90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c780 tx=0x7ff47800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff4777fe640 1 -- 172.21.15.50:0/1431040569 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478004b60 con 0x7ff48015e670 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff4777fe640 1 -- 172.21.15.50:0/1431040569 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff478004d00 con 0x7ff48015e670 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48009c4f0 con 0x7ff48015e670 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff4777fe640 1 -- 172.21.15.50:0/1431040569 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47800aa90 con 0x7ff48015e670 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.238+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48009cd90 con 0x7ff48015e670 2024-04-02T16:23:03.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.239+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4800756f0 con 0x7ff48015e670 2024-04-02T16:23:03.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.239+0000 7ff4777fe640 1 -- 172.21.15.50:0/1431040569 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff47800ac30 con 0x7ff48015e670 2024-04-02T16:23:03.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.240+0000 7ff4777fe640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff46006c8f0 0x7ff46006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.240+0000 7ff4777fe640 1 -- 172.21.15.50:0/1431040569 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff47808c270 con 0x7ff48015e670 2024-04-02T16:23:03.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.240+0000 7ff47ed9d640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff46006c8f0 0x7ff46006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.240+0000 7ff47ed9d640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff46006c8f0 0x7ff46006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff470036000 tx=0x7ff47003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.244+0000 7ff4777fe640 1 -- 172.21.15.50:0/1431040569 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff478020070 con 0x7ff48015e670 2024-04-02T16:23:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.429+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4801f3fc0 con 0x7ff48015e670 2024-04-02T16:23:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.431+0000 7ff4777fe640 1 -- 172.21.15.50:0/1431040569 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7ff47805a280 con 0x7ff48015e670 2024-04-02T16:23:03.432 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:03.432 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff46006c8f0 msgr2=0x7ff46006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff46006c8f0 0x7ff46006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff470036000 tx=0x7ff47003b040 comp rx=0 tx=0).stop 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 msgr2=0x7ff4800b6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b6e90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c780 tx=0x7ff47800cc40 comp rx=0 tx=0).stop 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 shutdown_connections 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff46006c8f0 0x7ff46006edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff48015e670 0x7ff4800b6e90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48009aa40 0x7ff48006fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 --2- 172.21.15.50:0/1431040569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48006cff0 0x7ff4801f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 >> 172.21.15.50:0/1431040569 conn(0x7ff4800b3b90 msgr2=0x7ff48015f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 shutdown_connections 2024-04-02T16:23:03.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.433+0000 7ff4850d3640 1 -- 172.21.15.50:0/1431040569 wait complete. 2024-04-02T16:23:03.445 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T16:23:03.445 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:23:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.545+0000 7f2221e41640 1 Processor -- start 2024-04-02T16:23:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.545+0000 7f2221e41640 1 -- start start 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.546+0000 7f2221e41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 0x7f221c0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.546+0000 7f2221e41640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f221c0b4760 0x7f221c0d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.546+0000 7f2221e41640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f221c0b4c70 con 0x7f221c0d2cb0 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.546+0000 7f2221e41640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f221c0b5880 con 0x7f221c0b4760 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.546+0000 7f2221e41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f221c0b4f50 con 0x7f221c15e650 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.546+0000 7f221bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f221c0d2cb0 0x7f221c1622e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47800/0 (socket says 172.21.15.50:47800) 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.546+0000 7f221bfff640 1 -- 172.21.15.50:0/3113247209 learned_addr learned my addr 172.21.15.50:0/3113247209 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:03.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221affd640 1 --2- 172.21.15.50:0/3113247209 >> v2:172.21.15.188:3300/0 conn(0x7f221c0b4760 0x7f221c0d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221b7fe640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 0x7f221c0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407180751 0 0) 0x7f221c0b4c70 con 0x7f221c0d2cb0 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2204003520 con 0x7f221c0d2cb0 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2231956248 0 0) 0x7f2204003520 con 0x7f221c0d2cb0 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 msgr2=0x7f221c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 0x7f221c0b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 >> v2:172.21.15.188:3300/0 conn(0x7f221c0b4760 msgr2=0x7f221c0d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 --2- 172.21.15.50:0/3113247209 >> v2:172.21.15.188:3300/0 conn(0x7f221c0b4760 0x7f221c0d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c0b5230 con 0x7f221c0d2cb0 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f22140019f0 con 0x7f221c0d2cb0 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 >> v1:172.21.15.50:6789/0 conn(0x7f221c0d2cb0 legacy=0x7f221c1622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040039b0 0x7f2204003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2204004800 0x7f2204004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 0x7f22040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f221c0b5880 con 0x7f2204004800 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f221c0b4f50 con 0x7f22040039b0 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f22040141d0 con 0x7f221c15e650 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221affd640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2204004800 0x7f2204004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221b7fe640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040039b0 0x7f2204003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221bfff640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 0x7f22040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221affd640 1 -- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 msgr2=0x7f22040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221affd640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c15e650 0x7f22040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221affd640 1 -- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040039b0 msgr2=0x7f2204003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221affd640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040039b0 0x7f2204003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221affd640 1 -- 172.21.15.50:0/3113247209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2204003520 con 0x7f2204004800 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.547+0000 7f221b7fe640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040039b0 0x7f2204003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:03.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.548+0000 7f221affd640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2204004800 0x7f2204004c80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2210009a90 tx=0x7f2204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dfabdb479c6bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.548+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210039070 con 0x7f2204004800 2024-04-02T16:23:03.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.548+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221002c6b0 con 0x7f2204004800 2024-04-02T16:23:03.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.548+0000 7f221a7fc640 1 -- 172.21.15.50:0/3113247209 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210009690 con 0x7f2204004800 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.549+0000 7f2221e41640 1 -- 172.21.15.50:0/3113247209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2204004800 msgr2=0x7f2204004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.549+0000 7f2221e41640 1 --2- 172.21.15.50:0/3113247209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2204004800 0x7f2204004c80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2210009a90 tx=0x7f2204005830 comp rx=0 tx=0).stop 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.549+0000 7f221bfff640 1 -- 172.21.15.50:0/3113247209 reap_dead start 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.549+0000 7f2221e41640 1 -- 172.21.15.50:0/3113247209 shutdown_connections 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.549+0000 7f2221e41640 1 -- 172.21.15.50:0/3113247209 >> 172.21.15.50:0/3113247209 conn(0x7f221c09d580 msgr2=0x7f221c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.549+0000 7f2221e41640 1 -- 172.21.15.50:0/3113247209 shutdown_connections 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.549+0000 7f2221e41640 1 -- 172.21.15.50:0/3113247209 wait complete. 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.550+0000 7f2221e41640 1 Processor -- start 2024-04-02T16:23:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.550+0000 7f2221e41640 1 -- start start 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f2221e41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f221c06d660 0x7f221c0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f2221e41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c159070 0x7f221c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f2221e41640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f221c0cb8c0 0x7f221c0b8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f2221e41640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f221c15ea30 con 0x7f221c0cb8c0 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f2221e41640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f221c0d2270 con 0x7f221c06d660 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f2221e41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f221c15ebb0 con 0x7f221c159070 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f221c0cb8c0 0x7f221c0b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f221c0cb8c0 0x7f221c0b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37136/0 (socket says 172.21.15.50:37136) 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 -- 172.21.15.50:0/252489116 learned_addr learned my addr 172.21.15.50:0/252489116 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221b7fe640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f221c06d660 0x7f221c0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 -- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c159070 msgr2=0x7f221c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221affd640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c159070 0x7f221c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c159070 0x7f221c0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 -- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f221c06d660 msgr2=0x7f221c0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f221c06d660 0x7f221c0cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 -- 172.21.15.50:0/252489116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c0b7e70 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221b7fe640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f221c06d660 0x7f221c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221affd640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c159070 0x7f221c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.551+0000 7f221bfff640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f221c0cb8c0 0x7f221c0b8910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f221c0b6e10 tx=0x7f221400e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.552+0000 7f2203fff640 1 -- 172.21.15.50:0/252489116 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400eaf0 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.552+0000 7f2203fff640 1 -- 172.21.15.50:0/252489116 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2214018070 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.552+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c0cdb60 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.552+0000 7f2203fff640 1 -- 172.21.15.50:0/252489116 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214013420 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.552+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c0b8110 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.553+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221c0756f0 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.553+0000 7f2203fff640 1 -- 172.21.15.50:0/252489116 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22140135c0 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.554+0000 7f2203fff640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f220806c8f0 0x7f220806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.554+0000 7f2203fff640 1 -- 172.21.15.50:0/252489116 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f221408c2f0 con 0x7f221c0cb8c0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.554+0000 7f221b7fe640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f220806c8f0 0x7f220806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.554+0000 7f221b7fe640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f220806c8f0 0x7f220806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f220c006d40 tx=0x7f220c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.558+0000 7f2203fff640 1 -- 172.21.15.50:0/252489116 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2214059cb0 con 0x7f221c0cb8c0 2024-04-02T16:23:03.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.781+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f221c072a30 con 0x7f221c0cb8c0 2024-04-02T16:23:03.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.781+0000 7f2203fff640 1 -- 172.21.15.50:0/252489116 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f221405dfb0 con 0x7f221c0cb8c0 2024-04-02T16:23:03.783 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:03.783 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:23:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f220806c8f0 msgr2=0x7f220806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f220806c8f0 0x7f220806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f220c006d40 tx=0x7f220c008040 comp rx=0 tx=0).stop 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f221c0cb8c0 msgr2=0x7f221c0b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f221c0cb8c0 0x7f221c0b8910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f221c0b6e10 tx=0x7f221400e160 comp rx=0 tx=0).stop 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 shutdown_connections 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f220806c8f0 0x7f220806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f221c0cb8c0 0x7f221c0b8910 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221c159070 0x7f221c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 --2- 172.21.15.50:0/252489116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f221c06d660 0x7f221c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 >> 172.21.15.50:0/252489116 conn(0x7f221c09d580 msgr2=0x7f221c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 shutdown_connections 2024-04-02T16:23:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.783+0000 7f2221e41640 1 -- 172.21.15.50:0/252489116 wait complete. 2024-04-02T16:23:03.795 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:23:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.896+0000 7fc6c6463640 1 Processor -- start 2024-04-02T16:23:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.896+0000 7fc6c6463640 1 -- start start 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6c6463640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 0x7fc6c006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6c6463640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c00d0210 con 0x7fc6c015e670 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6c6463640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c006d740 con 0x7fc6c009aa40 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6c6463640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c006da20 con 0x7fc6c00b4680 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6bffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fc6c015e670 0x7fc6c00d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47816/0 (socket says 172.21.15.50:47816) 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6bffff640 1 -- 172.21.15.50:0/1927862957 learned_addr learned my addr 172.21.15.50:0/1927862957 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236830860 0 0) 0x7fc6c00d0210 con 0x7fc6c015e670 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6a8003520 con 0x7fc6c015e670 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.897+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 0x7fc6c006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1497048295 0 0) 0x7fc6a8003520 con 0x7fc6c015e670 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 msgr2=0x7fc6c006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 0x7fc6c006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 >> v1:172.21.15.188:6789/0 conn(0x7fc6c009aa40 legacy=0x7fc6c006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c00d04b0 con 0x7fc6c015e670 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fc6b0001670 con 0x7fc6c015e670 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 >> v1:172.21.15.50:6789/0 conn(0x7fc6c015e670 legacy=0x7fc6c00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a80039e0 0x7fc6a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6a8004800 0x7fc6a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 0x7fc6a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c006da20 con 0x7fc6a8004800 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a800bb00 con 0x7fc6a80039e0 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a800bd00 con 0x7fc6c00b4680 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6a8004800 0x7fc6a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6bffff640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a80039e0 0x7fc6a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6c4a00640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 0x7fc6a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6bf7fe640 1 -- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 msgr2=0x7fc6a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 0x7fc6a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6bf7fe640 1 -- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a80039e0 msgr2=0x7fc6a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a80039e0 0x7fc6a8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6bf7fe640 1 -- 172.21.15.50:0/1927862957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a8003520 con 0x7fc6a8004800 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.898+0000 7fc6c4a00640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c00b4680 0x7fc6a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.899+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6a8004800 0x7fc6a8004c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009a90 tx=0x7fc6a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f19629ffacfca65 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.899+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4039070 con 0x7fc6a8004800 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.899+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b402c680 con 0x7fc6a8004800 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.899+0000 7fc6beffd640 1 -- 172.21.15.50:0/1927862957 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4009690 con 0x7fc6a8004800 2024-04-02T16:23:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.900+0000 7fc6c6463640 1 -- 172.21.15.50:0/1927862957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6a8004800 msgr2=0x7fc6a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.900+0000 7fc6c6463640 1 --2- 172.21.15.50:0/1927862957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6a8004800 0x7fc6a8004c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009a90 tx=0x7fc6a8005830 comp rx=0 tx=0).stop 2024-04-02T16:23:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.900+0000 7fc6c4a00640 1 -- 172.21.15.50:0/1927862957 reap_dead start 2024-04-02T16:23:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.900+0000 7fc6c6463640 1 -- 172.21.15.50:0/1927862957 shutdown_connections 2024-04-02T16:23:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.900+0000 7fc6c6463640 1 -- 172.21.15.50:0/1927862957 >> 172.21.15.50:0/1927862957 conn(0x7fc6c00b3b90 msgr2=0x7fc6c00b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.900+0000 7fc6c6463640 1 -- 172.21.15.50:0/1927862957 shutdown_connections 2024-04-02T16:23:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.900+0000 7fc6c6463640 1 -- 172.21.15.50:0/1927862957 wait complete. 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.901+0000 7fc6c6463640 1 Processor -- start 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.901+0000 7fc6c6463640 1 -- start start 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c6463640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a8004800 0x7fc6c00b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c6463640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6c015a600 0x7fc6c015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c6463640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c01583c0 0x7fc6c0158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c6463640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c00d16a0 con 0x7fc6c015a600 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c6463640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c00d0b00 con 0x7fc6a8004800 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c6463640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c006fcb0 con 0x7fc6c01583c0 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6c015a600 0x7fc6c015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6c015a600 0x7fc6c015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37156/0 (socket says 172.21.15.50:37156) 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 -- 172.21.15.50:0/559728401 learned_addr learned my addr 172.21.15.50:0/559728401 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c4a00640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c01583c0 0x7fc6c0158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bffff640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a8004800 0x7fc6c00b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 -- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c01583c0 msgr2=0x7fc6c0158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c01583c0 0x7fc6c0158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 -- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a8004800 msgr2=0x7fc6c00b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a8004800 0x7fc6c00b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 -- 172.21.15.50:0/559728401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c015b370 con 0x7fc6c015a600 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6c4a00640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c01583c0 0x7fc6c0158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bffff640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a8004800 0x7fc6c00b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bf7fe640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6c015a600 0x7fc6c015aa80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4000c00 tx=0x7fc6b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.902+0000 7fc6bcff9640 1 -- 172.21.15.50:0/559728401 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4042020 con 0x7fc6c015a600 2024-04-02T16:23:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.903+0000 7fc6bcff9640 1 -- 172.21.15.50:0/559728401 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b403a040 con 0x7fc6c015a600 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.903+0000 7fc6bcff9640 1 -- 172.21.15.50:0/559728401 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4039070 con 0x7fc6c015a600 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.903+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c01f4ad0 con 0x7fc6c015a600 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.903+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01f47d0 con 0x7fc6c015a600 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.903+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c00756f0 con 0x7fc6c015a600 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.904+0000 7fc6bcff9640 1 -- 172.21.15.50:0/559728401 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6b4045020 con 0x7fc6c015a600 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.904+0000 7fc6bcff9640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc6ac06c8f0 0x7fc6ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.904+0000 7fc6bcff9640 1 -- 172.21.15.50:0/559728401 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fc6b40ad4d0 con 0x7fc6c015a600 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.905+0000 7fc6bffff640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc6ac06c8f0 0x7fc6ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.905+0000 7fc6bffff640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc6ac06c8f0 0x7fc6ac06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0007d90 tx=0x7fc6b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:03.909+0000 7fc6bcff9640 1 -- 172.21.15.50:0/559728401 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc6b404c070 con 0x7fc6c015a600 2024-04-02T16:23:04.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.134+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6c00b8020 con 0x7fc6c015a600 2024-04-02T16:23:04.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.135+0000 7fc6bcff9640 1 -- 172.21.15.50:0/559728401 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7fc6b407b450 con 0x7fc6c015a600 2024-04-02T16:23:04.136 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:04.136 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/3543196660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":3543196660},{"type":"v1","addr":"172.21.15.50:6834","nonce":3543196660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/279610414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":279610414},{"type":"v1","addr":"172.21.15.188:6836","nonce":279610414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/206128465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":206128465},{"type":"v1","addr":"172.21.15.50:6835","nonce":206128465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:22:00.747313+0000","modified":"2024-04-02T16:22:22.833211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/2647829095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2647829095},{"type":"v1","addr":"172.21.15.188:6837","nonce":2647829095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-02T16:23:04.136 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 9 2024-04-02T16:23:04.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc6ac06c8f0 msgr2=0x7fc6ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc6ac06c8f0 0x7fc6ac06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0007d90 tx=0x7fc6b0009040 comp rx=0 tx=0).stop 2024-04-02T16:23:04.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6c015a600 msgr2=0x7fc6c015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6c015a600 0x7fc6c015aa80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4000c00 tx=0x7fc6b4004730 comp rx=0 tx=0).stop 2024-04-02T16:23:04.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 shutdown_connections 2024-04-02T16:23:04.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6c01583c0 0x7fc6c0158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc6ac06c8f0 0x7fc6ac06edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6c015a600 0x7fc6c015aa80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 --2- 172.21.15.50:0/559728401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a8004800 0x7fc6c00b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 >> 172.21.15.50:0/559728401 conn(0x7fc6c00b3b90 msgr2=0x7fc6c00d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:04.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 shutdown_connections 2024-04-02T16:23:04.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.137+0000 7fc6c6463640 1 -- 172.21.15.50:0/559728401 wait complete. 2024-04-02T16:23:04.150 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:23:04.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.251+0000 7ff43883c640 1 Processor -- start 2024-04-02T16:23:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.251+0000 7ff43883c640 1 -- start start 2024-04-02T16:23:04.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.252+0000 7ff43883c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff434169360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.252+0000 7ff43883c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43406c760 con 0x7ff43406d800 2024-04-02T16:23:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.252+0000 7ff43883c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43406ca40 con 0x7ff43415e370 2024-04-02T16:23:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.252+0000 7ff43883c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff43406d370 con 0x7ff43409b150 2024-04-02T16:23:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.252+0000 7ff431d9b640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff43406d800 0x7ff43409aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47826/0 (socket says 172.21.15.50:47826) 2024-04-02T16:23:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.252+0000 7ff431d9b640 1 -- 172.21.15.50:0/3242192050 learned_addr learned my addr 172.21.15.50:0/3242192050 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:04.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.252+0000 7ff432d9d640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff434169360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013712285 0 0) 0x7ff43406c760 con 0x7ff43406d800 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff420003520 con 0x7ff43406d800 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280624500 0 0) 0x7ff43406ca40 con 0x7ff43415e370 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff43406c760 con 0x7ff43415e370 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1522006568 0 0) 0x7ff420003520 con 0x7ff43406d800 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 msgr2=0x7ff434169360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff434169360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 >> v1:172.21.15.188:6789/0 conn(0x7ff43415e370 legacy=0x7ff4340d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43406cd20 con 0x7ff43406d800 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7ff41c001670 con 0x7ff43406d800 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff432d9d640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff434169360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 >> v1:172.21.15.50:6789/0 conn(0x7ff43406d800 legacy=0x7ff43409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff420003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4200040b0 0x7ff420004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff43406d370 con 0x7ff420004d40 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff42000ba70 con 0x7ff4200040b0 2024-04-02T16:23:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff42000bc70 con 0x7ff43409b150 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff432d9d640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff431d9b640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4200040b0 0x7ff420004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43259c640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff420003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff432d9d640 1 -- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 msgr2=0x7ff420003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff432d9d640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff420003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff432d9d640 1 -- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4200040b0 msgr2=0x7ff420004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff432d9d640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4200040b0 0x7ff420004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff432d9d640 1 -- 172.21.15.50:0/3242192050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff420003520 con 0x7ff420004d40 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.253+0000 7ff43259c640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43409b150 0x7ff420003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.254+0000 7ff432d9d640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00e320 tx=0x7ff420005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2b2103cc5fb666 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.254+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c009dc0 con 0x7ff420004d40 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.254+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42c02cd60 con 0x7ff420004d40 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.254+0000 7ff43159a640 1 -- 172.21.15.50:0/3242192050 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c003b90 con 0x7ff420004d40 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.255+0000 7ff43883c640 1 -- 172.21.15.50:0/3242192050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff420004d40 msgr2=0x7ff4200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.255+0000 7ff43883c640 1 --2- 172.21.15.50:0/3242192050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00e320 tx=0x7ff420005830 comp rx=0 tx=0).stop 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.255+0000 7ff432d9d640 1 -- 172.21.15.50:0/3242192050 reap_dead start 2024-04-02T16:23:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.255+0000 7ff43883c640 1 -- 172.21.15.50:0/3242192050 shutdown_connections 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.255+0000 7ff43883c640 1 -- 172.21.15.50:0/3242192050 >> 172.21.15.50:0/3242192050 conn(0x7ff4340b3ba0 msgr2=0x7ff4340b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.255+0000 7ff43883c640 1 -- 172.21.15.50:0/3242192050 shutdown_connections 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.255+0000 7ff43883c640 1 -- 172.21.15.50:0/3242192050 wait complete. 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.256+0000 7ff43883c640 1 Processor -- start 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.256+0000 7ff43883c640 1 -- start start 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43883c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff420004d40 0x7ff4340c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43883c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff43414b100 0x7ff4340b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43883c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43415a510 0x7ff43415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43883c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4340d1640 con 0x7ff43414b100 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43883c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4340d14c0 con 0x7ff420004d40 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43883c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff43415e750 con 0x7ff43415a510 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff43414b100 0x7ff4340b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff43414b100 0x7ff4340b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37182/0 (socket says 172.21.15.50:37182) 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 -- 172.21.15.50:0/101392660 learned_addr learned my addr 172.21.15.50:0/101392660 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43259c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff420004d40 0x7ff4340c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 -- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43415a510 msgr2=0x7ff43415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff432d9d640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43415a510 0x7ff43415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43415a510 0x7ff43415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 -- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff420004d40 msgr2=0x7ff4340c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff420004d40 0x7ff4340c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff431d9b640 1 -- 172.21.15.50:0/101392660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43414b580 con 0x7ff43414b100 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff43259c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff420004d40 0x7ff4340c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.257+0000 7ff432d9d640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43415a510 0x7ff43415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.258+0000 7ff431d9b640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff43414b100 0x7ff4340b32b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0072c0 tx=0x7ff41c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.258+0000 7ff42affd640 1 -- 172.21.15.50:0/101392660 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00f9c0 con 0x7ff43414b100 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.258+0000 7ff42affd640 1 -- 172.21.15.50:0/101392660 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c018070 con 0x7ff43414b100 2024-04-02T16:23:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.258+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4340c7a50 con 0x7ff43414b100 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.258+0000 7ff42affd640 1 -- 172.21.15.50:0/101392660 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00fd80 con 0x7ff43414b100 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.258+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4340c8010 con 0x7ff43414b100 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.259+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4340756f0 con 0x7ff43414b100 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.259+0000 7ff42affd640 1 -- 172.21.15.50:0/101392660 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff41c01d460 con 0x7ff43414b100 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.260+0000 7ff42affd640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff41406c8f0 0x7ff41406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.260+0000 7ff42affd640 1 -- 172.21.15.50:0/101392660 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff41c08bf70 con 0x7ff43414b100 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.260+0000 7ff43259c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff41406c8f0 0x7ff41406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.260+0000 7ff43259c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff41406c8f0 0x7ff41406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff424007ae0 tx=0x7ff424009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.264+0000 7ff42affd640 1 -- 172.21.15.50:0/101392660 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff41c05a010 con 0x7ff43414b100 2024-04-02T16:23:04.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.451+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4340b8030 con 0x7ff43414b100 2024-04-02T16:23:04.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.452+0000 7ff42affd640 1 -- 172.21.15.50:0/101392660 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18876 (secure 0 0 0) 0x7ff41c059910 con 0x7ff43414b100 2024-04-02T16:23:04.453 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:04.453 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":27,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:22:41.733701+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:21:58.747655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:21:59.703020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T16:22:03.421200+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:23:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff41406c8f0 msgr2=0x7ff41406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff41406c8f0 0x7ff41406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff424007ae0 tx=0x7ff424009040 comp rx=0 tx=0).stop 2024-04-02T16:23:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff43414b100 msgr2=0x7ff4340b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff43414b100 0x7ff4340b32b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0072c0 tx=0x7ff41c007780 comp rx=0 tx=0).stop 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 shutdown_connections 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff43415a510 0x7ff43415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff41406c8f0 0x7ff41406edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff43414b100 0x7ff4340b32b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 --2- 172.21.15.50:0/101392660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff420004d40 0x7ff4340c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 >> 172.21.15.50:0/101392660 conn(0x7ff4340b3ba0 msgr2=0x7ff4340aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 shutdown_connections 2024-04-02T16:23:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.454+0000 7ff43883c640 1 -- 172.21.15.50:0/101392660 wait complete. 2024-04-02T16:23:04.467 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T16:23:04.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.568+0000 7f76b56d6640 1 Processor -- start 2024-04-02T16:23:04.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.568+0000 7f76b56d6640 1 -- start start 2024-04-02T16:23:04.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76b56d6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76b015e3d0 0x7f76b00d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76b56d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 0x7f76b00d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76b56d6640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b00d2f30 con 0x7f76b015e7b0 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76b56d6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f76b00cf000 con 0x7f76b015e3d0 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76b56d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f76b00cf2e0 con 0x7f76b00c7e30 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76af7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f76b015e7b0 0x7f76b00d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:47842/0 (socket says 172.21.15.50:47842) 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76af7fe640 1 -- 172.21.15.50:0/252139914 learned_addr learned my addr 172.21.15.50:0/252139914 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76ae7fc640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 0x7f76b00d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76aeffd640 1 --2- 172.21.15.50:0/252139914 >> v2:172.21.15.188:3300/0 conn(0x7f76b015e3d0 0x7f76b00d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922335296 0 0) 0x7f76b00d2f30 con 0x7f76b015e7b0 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7698003520 con 0x7f76b015e7b0 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2849937670 0 0) 0x7f7698003520 con 0x7f76b015e7b0 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 msgr2=0x7f76b00d4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 0x7f76b00d4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 >> v2:172.21.15.188:3300/0 conn(0x7f76b015e3d0 msgr2=0x7f76b00d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 --2- 172.21.15.50:0/252139914 >> v2:172.21.15.188:3300/0 conn(0x7f76b015e3d0 0x7f76b00d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b00cf5c0 con 0x7f76b015e7b0 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f76a00019f0 con 0x7f76b015e7b0 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 >> v1:172.21.15.50:6789/0 conn(0x7f76b015e7b0 legacy=0x7f76b00d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 0x7f76980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7698003f30 0x7f7698004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.569+0000 7f76adffb640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7698004d40 0x7f76980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00cf000 con 0x7f7698004d40 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00cf2e0 con 0x7f7698003f30 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f769800b9c0 con 0x7f76b00c7e30 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76af7fe640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7698004d40 0x7f76980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76aeffd640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 0x7f76980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76ae7fc640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7698003f30 0x7f7698004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76af7fe640 1 -- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 msgr2=0x7f76980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76af7fe640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b00c7e30 0x7f76980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76af7fe640 1 -- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7698003f30 msgr2=0x7f7698004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76af7fe640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7698003f30 0x7f7698004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.570+0000 7f76af7fe640 1 -- 172.21.15.50:0/252139914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7698003520 con 0x7f7698004d40 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76af7fe640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7698004d40 0x7f76980051c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f76a00013d0 tx=0x7f7698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4447a64b66cfbf50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a000a4e0 con 0x7f7698004d40 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0034650 con 0x7f7698004d40 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76adffb640 1 -- 172.21.15.50:0/252139914 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0034a40 con 0x7f7698004d40 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76b56d6640 1 -- 172.21.15.50:0/252139914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7698004d40 msgr2=0x7f76980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76b56d6640 1 --2- 172.21.15.50:0/252139914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7698004d40 0x7f76980051c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f76a00013d0 tx=0x7f7698005830 comp rx=0 tx=0).stop 2024-04-02T16:23:04.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76af7fe640 1 -- 172.21.15.50:0/252139914 reap_dead start 2024-04-02T16:23:04.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76b56d6640 1 -- 172.21.15.50:0/252139914 shutdown_connections 2024-04-02T16:23:04.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76b56d6640 1 -- 172.21.15.50:0/252139914 >> 172.21.15.50:0/252139914 conn(0x7f76b008dc90 msgr2=0x7f76b009d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:04.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76b56d6640 1 -- 172.21.15.50:0/252139914 shutdown_connections 2024-04-02T16:23:04.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.571+0000 7f76b56d6640 1 -- 172.21.15.50:0/252139914 wait complete. 2024-04-02T16:23:04.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 Processor -- start 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 -- start start 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b014e5f0 0x7f76b01e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b014e9d0 0x7f76b00cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b0160240 0x7f76b01606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00d2a00 con 0x7f76b0160240 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00d2880 con 0x7f76b014e9d0 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76b56d6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f76b00d2b80 con 0x7f76b014e5f0 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76af7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b0160240 0x7f76b01606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76af7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b0160240 0x7f76b01606c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37196/0 (socket says 172.21.15.50:37196) 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.573+0000 7f76af7fe640 1 -- 172.21.15.50:0/334470729 learned_addr learned my addr 172.21.15.50:0/334470729 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76aeffd640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b014e5f0 0x7f76b01e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76ae7fc640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b014e9d0 0x7f76b00cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76af7fe640 1 -- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b014e5f0 msgr2=0x7f76b01e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76af7fe640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b014e5f0 0x7f76b01e7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76af7fe640 1 -- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b014e9d0 msgr2=0x7f76b00cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76af7fe640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b014e9d0 0x7f76b00cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76af7fe640 1 -- 172.21.15.50:0/334470729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b01e8530 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76af7fe640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b0160240 0x7f76b01606c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f76a00013d0 tx=0x7f76a0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76977fe640 1 -- 172.21.15.50:0/334470729 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0039040 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76977fe640 1 -- 172.21.15.50:0/334470729 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0043070 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b01e8810 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76977fe640 1 -- 172.21.15.50:0/334470729 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0034a20 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.574+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b01e8dd0 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.575+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b0075760 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.576+0000 7f76977fe640 1 -- 172.21.15.50:0/334470729 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76a0034bc0 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.576+0000 7f76977fe640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f769c06c8f0 0x7f769c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.576+0000 7f76977fe640 1 -- 172.21.15.50:0/334470729 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f76a00ade00 con 0x7f76b0160240 2024-04-02T16:23:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.576+0000 7f76aeffd640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f769c06c8f0 0x7f769c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.577+0000 7f76aeffd640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f769c06c8f0 0x7f769c06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009c40 tx=0x7f76a8009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.580+0000 7f76977fe640 1 -- 172.21.15.50:0/334470729 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f76a00019b0 con 0x7f76b0160240 2024-04-02T16:23:04.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.805+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f76b00cb170 con 0x7f76b0160240 2024-04-02T16:23:04.807 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-04-02T16:23:04.806+0000 7f36d8ed6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T16:23:04.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.807+0000 7f76977fe640 1 -- 172.21.15.50:0/334470729 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f76a007be40 con 0x7f76b0160240 2024-04-02T16:23:04.808 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T16:23:04.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.809+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f769c06c8f0 msgr2=0x7f769c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.809+0000 7f76b56d6640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f769c06c8f0 0x7f769c06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009c40 tx=0x7f76a8009190 comp rx=0 tx=0).stop 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b0160240 msgr2=0x7f76b01606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b0160240 0x7f76b01606c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f76a00013d0 tx=0x7f76a0001360 comp rx=0 tx=0).stop 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 shutdown_connections 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f769c06c8f0 0x7f769c06edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b0160240 0x7f76b01606c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b014e9d0 0x7f76b00cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 --2- 172.21.15.50:0/334470729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b014e5f0 0x7f76b01e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 >> 172.21.15.50:0/334470729 conn(0x7f76b008dc90 msgr2=0x7f76b00c9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 shutdown_connections 2024-04-02T16:23:04.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.810+0000 7f76b56d6640 1 -- 172.21.15.50:0/334470729 wait complete. 2024-04-02T16:23:04.814 INFO:tasks.ceph.mds.b.smithi188.stderr: -6767> 2024-04-02T16:21:54.240+0000 7fb456c8e640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-04-02T16:23:04.815 INFO:tasks.ceph.mds.b.smithi188.stderr: -5539> 2024-04-02T16:22:05.236+0000 7fb450c82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T16:23:04.824 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T16:23:04.847 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-02T16:23:04.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.927+0000 7f17c4d60640 1 Processor -- start 2024-04-02T16:23:04.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.927+0000 7f17c4d60640 1 -- start start 2024-04-02T16:23:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17c4d60640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c009aa40 0x7f17c0070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17c4d60640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f17c00b4700 0x7f17c009ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17c4d60640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c00d1e60 con 0x7f17c006f560 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17c4d60640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f17c00b19c0 con 0x7f17c00b4700 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17c4d60640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f17c00d1670 con 0x7f17c009aa40 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17b7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f17c00b4700 0x7f17c009ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17be59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c009aa40 0x7f17c0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17bed9d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f17c006f560 0x7f17c006f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:60348/0 (socket says 172.21.15.50:60348) 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17bed9d640 1 -- 172.21.15.50:0/2811150917 learned_addr learned my addr 172.21.15.50:0/2811150917 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17b7fff640 1 -- 172.21.15.50:0/2811150917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c009aa40 msgr2=0x7f17c0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17b7fff640 1 --2- 172.21.15.50:0/2811150917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c009aa40 0x7f17c0070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17b7fff640 1 -- 172.21.15.50:0/2811150917 >> v1:172.21.15.188:6789/0 conn(0x7f17c006f560 legacy=0x7f17c006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.928+0000 7f17b7fff640 1 -- 172.21.15.50:0/2811150917 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c009b360 con 0x7f17c00b4700 2024-04-02T16:23:04.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.929+0000 7f17b7fff640 1 --2- 172.21.15.50:0/2811150917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 0x7f17c009ae20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f17a80097a0 tx=0x7f17a802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ed4cd11ae9ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.929+0000 7f17bdd9b640 1 -- 172.21.15.50:0/2811150917 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8035020 con 0x7f17c00b4700 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.929+0000 7f17bdd9b640 1 -- 172.21.15.50:0/2811150917 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a8031830 con 0x7f17c00b4700 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.929+0000 7f17bdd9b640 1 -- 172.21.15.50:0/2811150917 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8031c00 con 0x7f17c00b4700 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 -- 172.21.15.50:0/2811150917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 msgr2=0x7f17c009ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 --2- 172.21.15.50:0/2811150917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 0x7f17c009ae20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f17a80097a0 tx=0x7f17a802b8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 -- 172.21.15.50:0/2811150917 shutdown_connections 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 --2- 172.21.15.50:0/2811150917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 0x7f17c009ae20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 --2- 172.21.15.50:0/2811150917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c009aa40 0x7f17c0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 -- 172.21.15.50:0/2811150917 >> 172.21.15.50:0/2811150917 conn(0x7f17c00b3be0 msgr2=0x7f17c00b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 -- 172.21.15.50:0/2811150917 shutdown_connections 2024-04-02T16:23:04.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.930+0000 7f17c4d60640 1 -- 172.21.15.50:0/2811150917 wait complete. 2024-04-02T16:23:04.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.931+0000 7f17c4d60640 1 Processor -- start 2024-04-02T16:23:04.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.931+0000 7f17c4d60640 1 -- start start 2024-04-02T16:23:04.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17c4d60640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c006f560 0x7f17c015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17c4d60640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c009aa40 0x7f17c0148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17c4d60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 0x7f17c00afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17c4d60640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17c00710c0 con 0x7f17c009aa40 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17c4d60640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17c0070de0 con 0x7f17c00b4700 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17c4d60640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f17c006c6d0 con 0x7f17c006f560 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c009aa40 0x7f17c0148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c009aa40 0x7f17c0148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37208/0 (socket says 172.21.15.50:37208) 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 -- 172.21.15.50:0/1441184521 learned_addr learned my addr 172.21.15.50:0/1441184521 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17bed9d640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 0x7f17c00afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17be59c640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c006f560 0x7f17c015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 -- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c006f560 msgr2=0x7f17c015ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c006f560 0x7f17c015ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 -- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 msgr2=0x7f17c00afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:04.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 0x7f17c00afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:04.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.932+0000 7f17b7fff640 1 -- 172.21.15.50:0/1441184521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c0149010 con 0x7f17c009aa40 2024-04-02T16:23:04.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.933+0000 7f17b7fff640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c009aa40 0x7f17c0148ad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f17a80098c0 tx=0x7f17a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.933+0000 7f17b6ffd640 1 -- 172.21.15.50:0/1441184521 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8035040 con 0x7f17c009aa40 2024-04-02T16:23:04.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.933+0000 7f17b6ffd640 1 -- 172.21.15.50:0/1441184521 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a803b920 con 0x7f17c009aa40 2024-04-02T16:23:04.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.933+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c00b6b20 con 0x7f17c009aa40 2024-04-02T16:23:04.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.933+0000 7f17b6ffd640 1 -- 172.21.15.50:0/1441184521 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a803bd10 con 0x7f17c009aa40 2024-04-02T16:23:04.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.933+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c00b70e0 con 0x7f17c009aa40 2024-04-02T16:23:04.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.934+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c00b76f0 con 0x7f17c009aa40 2024-04-02T16:23:04.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.934+0000 7f17b6ffd640 1 -- 172.21.15.50:0/1441184521 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17a8038020 con 0x7f17c009aa40 2024-04-02T16:23:04.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.934+0000 7f17b6ffd640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f17a006c8f0 0x7f17a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:04.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.935+0000 7f17b6ffd640 1 -- 172.21.15.50:0/1441184521 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6850+0+0 (secure 0 0 0) 0x7f17a80adf50 con 0x7f17c009aa40 2024-04-02T16:23:04.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.935+0000 7f17be59c640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f17a006c8f0 0x7f17a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:04.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.935+0000 7f17be59c640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f17a006c8f0 0x7f17a006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009b40 tx=0x7f17b0009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:04.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:04.939+0000 7f17b6ffd640 1 -- 172.21.15.50:0/1441184521 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f17a807b940 con 0x7f17c009aa40 2024-04-02T16:23:05.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.161+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f17c0075760 con 0x7f17c009aa40 2024-04-02T16:23:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.822+0000 7f17b6ffd640 1 -- 172.21.15.50:0/1441184521 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f17a807fc40 con 0x7f17c009aa40 2024-04-02T16:23:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f17a006c8f0 msgr2=0x7f17a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f17a006c8f0 0x7f17a006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009b40 tx=0x7f17b0009190 comp rx=0 tx=0).stop 2024-04-02T16:23:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c009aa40 msgr2=0x7f17c0148ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c009aa40 0x7f17c0148ad0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f17a80098c0 tx=0x7f17a80046c0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 shutdown_connections 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17c00b4700 0x7f17c00afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f17a006c8f0 0x7f17a006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c009aa40 0x7f17c0148ad0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 --2- 172.21.15.50:0/1441184521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17c006f560 0x7f17c015ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 >> 172.21.15.50:0/1441184521 conn(0x7f17c00b3be0 msgr2=0x7f17c00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 shutdown_connections 2024-04-02T16:23:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.824+0000 7f17c4d60640 1 -- 172.21.15.50:0/1441184521 wait complete. 2024-04-02T16:23:05.838 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T16:23:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.947+0000 7fb43c15f640 1 Processor -- start 2024-04-02T16:23:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.947+0000 7fb43c15f640 1 -- start start 2024-04-02T16:23:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43c15f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb4340cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43c15f640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb4340d2fa0 0x7fb4340b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43c15f640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb434161f20 con 0x7fb4340b0e70 2024-04-02T16:23:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43c15f640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fb4340b4890 con 0x7fb4340d2fa0 2024-04-02T16:23:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43c15f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340b3c80 con 0x7fb434161b40 2024-04-02T16:23:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43a6fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fb4340d2fa0 0x7fb4340b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb4396fa640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb4340b0e70 0x7fb4340d2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43552/0 (socket says 172.21.15.50:43552) 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb4396fa640 1 -- 172.21.15.50:0/676923712 learned_addr learned my addr 172.21.15.50:0/676923712 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43a6fc640 1 -- 172.21.15.50:0/676923712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 msgr2=0x7fb4340cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb439efb640 1 --2- 172.21.15.50:0/676923712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb4340cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43a6fc640 1 --2- 172.21.15.50:0/676923712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb4340cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43a6fc640 1 -- 172.21.15.50:0/676923712 >> v1:172.21.15.50:6789/0 conn(0x7fb4340b0e70 legacy=0x7fb4340d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb43a6fc640 1 -- 172.21.15.50:0/676923712 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340b3f60 con 0x7fb4340d2fa0 2024-04-02T16:23:05.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.948+0000 7fb438ef9640 1 -- 172.21.15.50:0/676923712 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3561949272 0 0) 0x7fb434161f20 con 0x7fb4340b0e70 2024-04-02T16:23:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb43a6fc640 1 --2- 172.21.15.50:0/676923712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340d2fa0 0x7fb4340b4d20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009d90 tx=0x7fb42c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0824bc31293bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb438ef9640 1 -- 172.21.15.50:0/676923712 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c039070 con 0x7fb4340d2fa0 2024-04-02T16:23:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb438ef9640 1 -- 172.21.15.50:0/676923712 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42c02cc50 con 0x7fb4340d2fa0 2024-04-02T16:23:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb438ef9640 1 -- 172.21.15.50:0/676923712 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c034440 con 0x7fb4340d2fa0 2024-04-02T16:23:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb43c15f640 1 -- 172.21.15.50:0/676923712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340d2fa0 msgr2=0x7fb4340b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb43c15f640 1 --2- 172.21.15.50:0/676923712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340d2fa0 0x7fb4340b4d20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009d90 tx=0x7fb42c02bea0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb43c15f640 1 -- 172.21.15.50:0/676923712 shutdown_connections 2024-04-02T16:23:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb43c15f640 1 --2- 172.21.15.50:0/676923712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340d2fa0 0x7fb4340b4d20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.949+0000 7fb43c15f640 1 --2- 172.21.15.50:0/676923712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb4340cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.950+0000 7fb43c15f640 1 -- 172.21.15.50:0/676923712 >> 172.21.15.50:0/676923712 conn(0x7fb43409d580 msgr2=0x7fb4340d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.950+0000 7fb43c15f640 1 -- 172.21.15.50:0/676923712 shutdown_connections 2024-04-02T16:23:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.950+0000 7fb43c15f640 1 -- 172.21.15.50:0/676923712 wait complete. 2024-04-02T16:23:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.951+0000 7fb43c15f640 1 Processor -- start 2024-04-02T16:23:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.951+0000 7fb43c15f640 1 -- start start 2024-04-02T16:23:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.951+0000 7fb43c15f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340b0e70 0x7fb434154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.951+0000 7fb43c15f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4340d2fa0 0x7fb434153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.951+0000 7fb43c15f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb43415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb43c15f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340d2350 con 0x7fb4340b0e70 2024-04-02T16:23:05.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb43c15f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340b4520 con 0x7fb4340d2fa0 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb43c15f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340cfea0 con 0x7fb434161b40 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340b0e70 0x7fb434154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340b0e70 0x7fb434154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35194/0 (socket says 172.21.15.50:35194) 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 -- 172.21.15.50:0/3665260637 learned_addr learned my addr 172.21.15.50:0/3665260637 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 -- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 msgr2=0x7fb43415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb43a6fc640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb43415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb43415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 -- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4340d2fa0 msgr2=0x7fb434153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4340d2fa0 0x7fb434153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 -- 172.21.15.50:0/3665260637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb434159b20 con 0x7fb4340b0e70 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb439efb640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340b0e70 0x7fb434154530 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c7f0 tx=0x7fb43000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb42a7fc640 1 -- 172.21.15.50:0/3665260637 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430015070 con 0x7fb4340b0e70 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb42a7fc640 1 -- 172.21.15.50:0/3665260637 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb430004220 con 0x7fb4340b0e70 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb42a7fc640 1 -- 172.21.15.50:0/3665260637 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430007620 con 0x7fb4340b0e70 2024-04-02T16:23:05.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb434155b40 con 0x7fb4340b0e70 2024-04-02T16:23:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.952+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4341563e0 con 0x7fb4340b0e70 2024-04-02T16:23:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.953+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4340757d0 con 0x7fb4340b0e70 2024-04-02T16:23:05.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.954+0000 7fb42a7fc640 1 -- 172.21.15.50:0/3665260637 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4300079d0 con 0x7fb4340b0e70 2024-04-02T16:23:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.954+0000 7fb42a7fc640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb41806c8f0 0x7fb41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.954+0000 7fb42a7fc640 1 -- 172.21.15.50:0/3665260637 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6850+0+0 (secure 0 0 0) 0x7fb43008c370 con 0x7fb4340b0e70 2024-04-02T16:23:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.954+0000 7fb4396fa640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb41806c8f0 0x7fb41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:05.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.955+0000 7fb4396fa640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb41806c8f0 0x7fb41806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb424007fd0 tx=0x7fb424007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:05.959+0000 7fb42a7fc640 1 -- 172.21.15.50:0/3665260637 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb43005a320 con 0x7fb4340b0e70 2024-04-02T16:23:06.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.144+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb434155d10 con 0x7fb4340b0e70 2024-04-02T16:23:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.826+0000 7fb42a7fc640 1 -- 172.21.15.50:0/3665260637 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7fb430059c20 con 0x7fb4340b0e70 2024-04-02T16:23:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-04-02T16:23:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb41806c8f0 msgr2=0x7fb41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb41806c8f0 0x7fb41806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb424007fd0 tx=0x7fb424007920 comp rx=0 tx=0).stop 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340b0e70 msgr2=0x7fb434154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340b0e70 0x7fb434154530 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c7f0 tx=0x7fb43000ccb0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 shutdown_connections 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb434161b40 0x7fb43415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4340d2fa0 0x7fb434153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb41806c8f0 0x7fb41806edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 --2- 172.21.15.50:0/3665260637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb4340b0e70 0x7fb434154530 secure :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c7f0 tx=0x7fb43000ccb0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 >> 172.21.15.50:0/3665260637 conn(0x7fb43409d580 msgr2=0x7fb434160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 shutdown_connections 2024-04-02T16:23:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.829+0000 7fb43c15f640 1 -- 172.21.15.50:0/3665260637 wait complete. 2024-04-02T16:23:06.843 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T16:23:06.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.950+0000 7f0ddbd1c640 1 Processor -- start 2024-04-02T16:23:06.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.950+0000 7f0ddbd1c640 1 -- start start 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0ddbd1c640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0dd415c2d0 0x7f0dd406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0ddbd1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd409b150 0x7f0dd40cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0ddbd1c640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd40d0630 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0ddbd1c640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd40d0910 con 0x7f0dd415c2d0 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0ddbd1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd40d0bf0 con 0x7f0dd409b150 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0dd9ab8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0dd415c2d0 0x7f0dd406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0dd92b7640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f0dd40cc1e0 0x7f0dd409aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43556/0 (socket says 172.21.15.50:43556) 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.951+0000 7f0dd92b7640 1 -- 172.21.15.50:0/3713885593 learned_addr learned my addr 172.21.15.50:0/3713885593 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.952+0000 7f0dd9ab8640 1 -- 172.21.15.50:0/3713885593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd409b150 msgr2=0x7f0dd40cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.952+0000 7f0dd9ab8640 1 --2- 172.21.15.50:0/3713885593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd409b150 0x7f0dd40cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.952+0000 7f0dd9ab8640 1 -- 172.21.15.50:0/3713885593 >> v1:172.21.15.50:6789/0 conn(0x7f0dd40cc1e0 legacy=0x7f0dd409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:06.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.952+0000 7f0dd8ab6640 1 -- 172.21.15.50:0/3713885593 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1135857348 0 0) 0x7f0dd40d0630 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.952+0000 7f0dd9ab8640 1 -- 172.21.15.50:0/3713885593 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd40ccd00 con 0x7f0dd415c2d0 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.952+0000 7f0dd9ab8640 1 --2- 172.21.15.50:0/3713885593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd415c2d0 0x7f0dd406f5a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009d90 tx=0x7f0dc402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f976020b7f31f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0dd8ab6640 1 -- 172.21.15.50:0/3713885593 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4039070 con 0x7f0dd415c2d0 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0dd8ab6640 1 -- 172.21.15.50:0/3713885593 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc402cc50 con 0x7f0dd415c2d0 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0dd8ab6640 1 -- 172.21.15.50:0/3713885593 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4034440 con 0x7f0dd415c2d0 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/3713885593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd415c2d0 msgr2=0x7f0dd406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/3713885593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd415c2d0 0x7f0dd406f5a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009d90 tx=0x7f0dc402bea0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/3713885593 shutdown_connections 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/3713885593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd409b150 0x7f0dd40cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/3713885593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd415c2d0 0x7f0dd406f5a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/3713885593 >> 172.21.15.50:0/3713885593 conn(0x7f0dd40b3b40 msgr2=0x7f0dd40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:06.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/3713885593 shutdown_connections 2024-04-02T16:23:06.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.953+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/3713885593 wait complete. 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.954+0000 7f0ddbd1c640 1 Processor -- start 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.954+0000 7f0ddbd1c640 1 -- start start 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0ddbd1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dd409b150 0x7f0dd41567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0ddbd1c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd40cc1e0 0x7f0dd40b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0ddbd1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd415c2d0 0x7f0dd40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0ddbd1c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd415c6b0 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0ddbd1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd40d0ed0 con 0x7f0dd409b150 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0ddbd1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd415c830 con 0x7f0dd415c2d0 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd40cc1e0 0x7f0dd40b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd40cc1e0 0x7f0dd40b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35212/0 (socket says 172.21.15.50:35212) 2024-04-02T16:23:06.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 -- 172.21.15.50:0/2205451049 learned_addr learned my addr 172.21.15.50:0/2205451049 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 -- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd415c2d0 msgr2=0x7f0dd40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd415c2d0 0x7f0dd40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 -- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dd409b150 msgr2=0x7f0dd41567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dd409b150 0x7f0dd41567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.955+0000 7f0dd92b7640 1 -- 172.21.15.50:0/2205451049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd40b58c0 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.956+0000 7f0dd92b7640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd40cc1e0 0x7f0dd40b5070 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c4a0 tx=0x7f0dc800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.956+0000 7f0dd27fc640 1 -- 172.21.15.50:0/2205451049 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8017070 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.956+0000 7f0dd27fc640 1 -- 172.21.15.50:0/2205451049 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc8005ce0 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.956+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd40b5ba0 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.956+0000 7f0dd27fc640 1 -- 172.21.15.50:0/2205451049 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80072b0 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.956+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd40b6440 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.957+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd40b5f90 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.957+0000 7f0dd27fc640 1 -- 172.21.15.50:0/2205451049 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dc801a070 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.958+0000 7f0dd27fc640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0dc006c8f0 0x7f0dc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:06.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.958+0000 7f0dd27fc640 1 -- 172.21.15.50:0/2205451049 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6412+0+0 (secure 0 0 0) 0x7f0dc808b140 con 0x7f0dd40cc1e0 2024-04-02T16:23:06.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.958+0000 7f0dd9ab8640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0dc006c8f0 0x7f0dc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:06.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.958+0000 7f0dd9ab8640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0dc006c8f0 0x7f0dc006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4036000 tx=0x7f0dc402e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:06.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:06.962+0000 7f0dd27fc640 1 -- 172.21.15.50:0/2205451049 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0dc8059340 con 0x7f0dd40cc1e0 2024-04-02T16:23:07.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.151+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0dd40757d0 con 0x7f0dd40cc1e0 2024-04-02T16:23:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.827+0000 7f0dd27fc640 1 -- 172.21.15.50:0/2205451049 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f0dc8058c40 con 0x7f0dd40cc1e0 2024-04-02T16:23:07.829 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-04-02T16:23:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0dc006c8f0 msgr2=0x7f0dc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0dc006c8f0 0x7f0dc006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4036000 tx=0x7f0dc402e000 comp rx=0 tx=0).stop 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd40cc1e0 msgr2=0x7f0dd40b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd40cc1e0 0x7f0dd40b5070 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c4a0 tx=0x7f0dc800c960 comp rx=0 tx=0).stop 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 shutdown_connections 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd415c2d0 0x7f0dd40cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0dc006c8f0 0x7f0dc006edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dd40cc1e0 0x7f0dd40b5070 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 --2- 172.21.15.50:0/2205451049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dd409b150 0x7f0dd41567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.831+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 >> 172.21.15.50:0/2205451049 conn(0x7f0dd40b3b40 msgr2=0x7f0dd40b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.832+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 shutdown_connections 2024-04-02T16:23:07.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.832+0000 7f0ddbd1c640 1 -- 172.21.15.50:0/2205451049 wait complete. 2024-04-02T16:23:07.849 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-02T16:23:07.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.959+0000 7f0dff835640 1 Processor -- start 2024-04-02T16:23:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dff835640 1 -- start start 2024-04-02T16:23:07.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dff835640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0df8162570 0x7f0df80cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:07.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dff835640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0df80d0110 0x7f0df815e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:07.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dff835640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df80d0670 con 0x7f0df8162950 2024-04-02T16:23:07.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dff835640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f0df80d0950 con 0x7f0df80d0110 2024-04-02T16:23:07.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dff835640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0df80d0c30 con 0x7f0df8162570 2024-04-02T16:23:07.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dfcdd0640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f0df80d0110 0x7f0df815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:07.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dfddd2640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f0df8162950 0x7f0df815e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43560/0 (socket says 172.21.15.50:43560) 2024-04-02T16:23:07.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.960+0000 7f0dfddd2640 1 -- 172.21.15.50:0/4005675897 learned_addr learned my addr 172.21.15.50:0/4005675897 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:07.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.961+0000 7f0dfd5d1640 1 --2- 172.21.15.50:0/4005675897 >> v2:172.21.15.188:3300/0 conn(0x7f0df8162570 0x7f0df80cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:07.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.961+0000 7f0dfcdd0640 1 -- 172.21.15.50:0/4005675897 >> v2:172.21.15.188:3300/0 conn(0x7f0df8162570 msgr2=0x7f0df80cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:07.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.961+0000 7f0dfcdd0640 1 --2- 172.21.15.50:0/4005675897 >> v2:172.21.15.188:3300/0 conn(0x7f0df8162570 0x7f0df80cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.961+0000 7f0dfcdd0640 1 -- 172.21.15.50:0/4005675897 >> v1:172.21.15.50:6789/0 conn(0x7f0df8162950 legacy=0x7f0df815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:07.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.961+0000 7f0df7fff640 1 -- 172.21.15.50:0/4005675897 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322982032 0 0) 0x7f0df80d0670 con 0x7f0df8162950 2024-04-02T16:23:07.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.961+0000 7f0dfcdd0640 1 -- 172.21.15.50:0/4005675897 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df80d0f10 con 0x7f0df80d0110 2024-04-02T16:23:07.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dfcdd0640 1 --2- 172.21.15.50:0/4005675897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df80d0110 0x7f0df815e150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0df80ce8d0 tx=0x7f0dec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35314fab9c3b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:07.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0df7fff640 1 -- 172.21.15.50:0/4005675897 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec035020 con 0x7f0df80d0110 2024-04-02T16:23:07.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0df7fff640 1 -- 172.21.15.50:0/4005675897 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec031830 con 0x7f0df80d0110 2024-04-02T16:23:07.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0df7fff640 1 -- 172.21.15.50:0/4005675897 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec031c00 con 0x7f0df80d0110 2024-04-02T16:23:07.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 -- 172.21.15.50:0/4005675897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df80d0110 msgr2=0x7f0df815e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:07.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 --2- 172.21.15.50:0/4005675897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df80d0110 0x7f0df815e150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0df80ce8d0 tx=0x7f0dec02b8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 -- 172.21.15.50:0/4005675897 shutdown_connections 2024-04-02T16:23:07.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 --2- 172.21.15.50:0/4005675897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df80d0110 0x7f0df815e150 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 --2- 172.21.15.50:0/4005675897 >> v2:172.21.15.188:3300/0 conn(0x7f0df8162570 0x7f0df80cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 -- 172.21.15.50:0/4005675897 >> 172.21.15.50:0/4005675897 conn(0x7f0df809d5a0 msgr2=0x7f0df80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:07.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 -- 172.21.15.50:0/4005675897 shutdown_connections 2024-04-02T16:23:07.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.962+0000 7f0dff835640 1 -- 172.21.15.50:0/4005675897 wait complete. 2024-04-02T16:23:07.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 Processor -- start 2024-04-02T16:23:07.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 -- start start 2024-04-02T16:23:07.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df80d0110 0x7f0df80b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0df8162570 0x7f0df80b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df8162950 0x7f0df80b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8163160 con 0x7f0df8162950 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8162fe0 con 0x7f0df80d0110 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dff835640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0df80d1710 con 0x7f0df8162570 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dfddd2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df8162950 0x7f0df80b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dfddd2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df8162950 0x7f0df80b73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35228/0 (socket says 172.21.15.50:35228) 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dfddd2640 1 -- 172.21.15.50:0/1062462741 learned_addr learned my addr 172.21.15.50:0/1062462741 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.964+0000 7f0dfd5d1640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df80d0110 0x7f0df80b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dfcdd0640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0df8162570 0x7f0df80b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dfddd2640 1 -- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0df8162570 msgr2=0x7f0df80b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dfddd2640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0df8162570 0x7f0df80b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dfddd2640 1 -- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df80d0110 msgr2=0x7f0df80b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dfddd2640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df80d0110 0x7f0df80b8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:07.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dfddd2640 1 -- 172.21.15.50:0/1062462741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df80b91c0 con 0x7f0df8162950 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dfddd2640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df8162950 0x7f0df80b73f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0de800c7e0 tx=0x7f0de800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0df5ffb640 1 -- 172.21.15.50:0/1062462741 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8018070 con 0x7f0df8162950 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0df5ffb640 1 -- 172.21.15.50:0/1062462741 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de800a5e0 con 0x7f0df8162950 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0df5ffb640 1 -- 172.21.15.50:0/1062462741 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de800a9d0 con 0x7f0df8162950 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df80b7930 con 0x7f0df8162950 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.965+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df80b7ec0 con 0x7f0df8162950 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.966+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df80757d0 con 0x7f0df8162950 2024-04-02T16:23:07.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.967+0000 7f0df5ffb640 1 -- 172.21.15.50:0/1062462741 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0de801b070 con 0x7f0df8162950 2024-04-02T16:23:07.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.967+0000 7f0df5ffb640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0de406c8f0 0x7f0de406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:07.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.967+0000 7f0df5ffb640 1 -- 172.21.15.50:0/1062462741 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f0de808b0a0 con 0x7f0df8162950 2024-04-02T16:23:07.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.967+0000 7f0dfd5d1640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0de406c8f0 0x7f0de406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:07.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.968+0000 7f0dfd5d1640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0de406c8f0 0x7f0de406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0df0006fd0 tx=0x7f0df0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:07.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:07.972+0000 7f0df5ffb640 1 -- 172.21.15.50:0/1062462741 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0de8058e80 con 0x7f0df8162950 2024-04-02T16:23:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.155+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0df80d1ac0 con 0x7f0df8162950 2024-04-02T16:23:08.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.842+0000 7f0df5ffb640 1 -- 172.21.15.50:0/1062462741 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f0de805d180 con 0x7f0df8162950 2024-04-02T16:23:08.844 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' removed 2024-04-02T16:23:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0de406c8f0 msgr2=0x7f0de406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0de406c8f0 0x7f0de406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0df0006fd0 tx=0x7f0df0008040 comp rx=0 tx=0).stop 2024-04-02T16:23:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df8162950 msgr2=0x7f0df80b73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df8162950 0x7f0df80b73f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0de800c7e0 tx=0x7f0de800cca0 comp rx=0 tx=0).stop 2024-04-02T16:23:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 shutdown_connections 2024-04-02T16:23:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f0de406c8f0 0x7f0de406edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df8162950 0x7f0df80b73f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0df8162570 0x7f0df80b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 --2- 172.21.15.50:0/1062462741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0df80d0110 0x7f0df80b8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 >> 172.21.15.50:0/1062462741 conn(0x7f0df809d5a0 msgr2=0x7f0df806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 shutdown_connections 2024-04-02T16:23:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:08.846+0000 7f0dff835640 1 -- 172.21.15.50:0/1062462741 wait complete. 2024-04-02T16:23:08.860 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-02T16:23:08.860 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-02T16:23:08.861 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-02T16:23:08.861 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:23:08.861 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-02T16:23:08.861 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-02T16:23:08.861 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-02T16:23:08.861 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:23:08.861 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-02T16:23:08.862 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-02T16:23:08.862 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-02T16:23:08.862 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:23:08.862 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-02T16:23:08.862 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-02T16:23:08.862 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-02T16:23:08.862 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:23:08.873 INFO:tasks.ceph.mds.a:Stopped 2024-04-02T16:23:08.873 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-02T16:23:08.874 INFO:tasks.ceph.mds.c:Stopped 2024-04-02T16:23:08.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-02T16:23:08.876 INFO:tasks.ceph.mds.b:Stopped 2024-04-02T16:23:08.876 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-02T16:23:08.877 INFO:tasks.ceph.mds.d:Stopped 2024-04-02T16:23:08.877 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-02T16:23:08.917 INFO:tasks.ceph.mds.a:Started 2024-04-02T16:23:08.918 INFO:tasks.ceph.mds.c:Started 2024-04-02T16:23:08.918 DEBUG:teuthology.parallel:result is None 2024-04-02T16:23:08.918 DEBUG:teuthology.parallel:result is None 2024-04-02T16:23:08.920 INFO:tasks.ceph.mds.b:Started 2024-04-02T16:23:08.921 DEBUG:teuthology.parallel:result is None 2024-04-02T16:23:08.921 INFO:tasks.ceph.mds.d:Started 2024-04-02T16:23:08.921 DEBUG:teuthology.parallel:result is None 2024-04-02T16:23:08.922 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-02T16:23:08.996 INFO:tasks.ceph.mds.a.smithi050.stdout:starting mds.a at 2024-04-02T16:23:08.997 INFO:tasks.ceph.mds.c.smithi050.stdout:starting mds.c at 2024-04-02T16:23:09.003 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-02T16:23:09.003 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-02T16:23:09.004 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-02T16:23:09.003+0000 7f35a967d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-02T16:23:09.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.039+0000 7fbdb5dc8640 1 Processor -- start 2024-04-02T16:23:09.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.039+0000 7fbdb5dc8640 1 -- start start 2024-04-02T16:23:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdb5dc8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbdb0161c80 0x7fbdb00d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdb5dc8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbdb00d2e50 0x7fbdb00d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdb5dc8640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb00b10e0 con 0x7fbdb00d1200 2024-04-02T16:23:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdb5dc8640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb00b1380 con 0x7fbdb00d2e50 2024-04-02T16:23:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdb5dc8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb00b1660 con 0x7fbdb0161c80 2024-04-02T16:23:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaeffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fbdb00d2e50 0x7fbdb00d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaffff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fbdb00d1200 0x7fbdb00cef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43584/0 (socket says 172.21.15.50:43584) 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaffff640 1 -- 172.21.15.50:0/2364748405 learned_addr learned my addr 172.21.15.50:0/2364748405 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdae7fc640 1 -- 172.21.15.50:0/2364748405 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588423355 0 0) 0x7fbdb00b10e0 con 0x7fbdb00d1200 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdae7fc640 1 -- 172.21.15.50:0/2364748405 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd98003520 con 0x7fbdb00d1200 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaeffd640 1 -- 172.21.15.50:0/2364748405 >> v2:172.21.15.188:3300/0 conn(0x7fbdb0161c80 msgr2=0x7fbdb00d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaf7fe640 1 --2- 172.21.15.50:0/2364748405 >> v2:172.21.15.188:3300/0 conn(0x7fbdb0161c80 0x7fbdb00d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaeffd640 1 --2- 172.21.15.50:0/2364748405 >> v2:172.21.15.188:3300/0 conn(0x7fbdb0161c80 0x7fbdb00d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaeffd640 1 -- 172.21.15.50:0/2364748405 >> v1:172.21.15.50:6789/0 conn(0x7fbdb00d1200 legacy=0x7fbdb00cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaeffd640 1 -- 172.21.15.50:0/2364748405 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb00d3340 con 0x7fbdb00d2e50 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.040+0000 7fbdaf7fe640 1 --2- 172.21.15.50:0/2364748405 >> v2:172.21.15.188:3300/0 conn(0x7fbdb0161c80 0x7fbdb00d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.041+0000 7fbdaeffd640 1 --2- 172.21.15.50:0/2364748405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb00d2e50 0x7fbdb00d0cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbdb008ea10 tx=0x7fbda402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892af377a102df49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.041+0000 7fbdae7fc640 1 -- 172.21.15.50:0/2364748405 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda40315a0 con 0x7fbdb00d2e50 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.041+0000 7fbdae7fc640 1 -- 172.21.15.50:0/2364748405 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda4037070 con 0x7fbdb00d2e50 2024-04-02T16:23:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.041+0000 7fbdae7fc640 1 -- 172.21.15.50:0/2364748405 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda40315a0 con 0x7fbdb00d2e50 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/2364748405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb00d2e50 msgr2=0x7fbdb00d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/2364748405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb00d2e50 0x7fbdb00d0cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbdb008ea10 tx=0x7fbda402b840 comp rx=0 tx=0).stop 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/2364748405 shutdown_connections 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/2364748405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb00d2e50 0x7fbdb00d0cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/2364748405 >> v2:172.21.15.188:3300/0 conn(0x7fbdb0161c80 0x7fbdb00d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/2364748405 >> 172.21.15.50:0/2364748405 conn(0x7fbdb009d5e0 msgr2=0x7fbdb00d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/2364748405 shutdown_connections 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.042+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/2364748405 wait complete. 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.043+0000 7fbdb5dc8640 1 Processor -- start 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.043+0000 7fbdb5dc8640 1 -- start start 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdb5dc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb00d1200 0x7fbdb0152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdb5dc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdb00d2e50 0x7fbdb00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdb5dc8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb0161c80 0x7fbdb006c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdb5dc8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb00b2e70 con 0x7fbdb0161c80 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdb5dc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb00d0390 con 0x7fbdb00d1200 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdb5dc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb00b2ff0 con 0x7fbdb00d2e50 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb0161c80 0x7fbdb006c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb0161c80 0x7fbdb006c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35270/0 (socket says 172.21.15.50:35270) 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 -- 172.21.15.50:0/3060965961 learned_addr learned my addr 172.21.15.50:0/3060965961 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaf7fe640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb00d1200 0x7fbdb0152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaeffd640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdb00d2e50 0x7fbdb00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 -- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdb00d2e50 msgr2=0x7fbdb00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdb00d2e50 0x7fbdb00cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 -- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb00d1200 msgr2=0x7fbdb0152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb00d1200 0x7fbdb0152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 -- 172.21.15.50:0/3060965961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb0152cd0 con 0x7fbdb0161c80 2024-04-02T16:23:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.044+0000 7fbdaffff640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb0161c80 0x7fbdb006c2c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbda800caa0 tx=0x7fbda800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.045+0000 7fbd97fff640 1 -- 172.21.15.50:0/3060965961 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8017070 con 0x7fbdb0161c80 2024-04-02T16:23:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.045+0000 7fbd97fff640 1 -- 172.21.15.50:0/3060965961 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda800ac00 con 0x7fbdb0161c80 2024-04-02T16:23:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.045+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb01f77f0 con 0x7fbdb0161c80 2024-04-02T16:23:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.045+0000 7fbd97fff640 1 -- 172.21.15.50:0/3060965961 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8012400 con 0x7fbdb0161c80 2024-04-02T16:23:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.045+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb01f7db0 con 0x7fbdb0161c80 2024-04-02T16:23:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.046+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb0075760 con 0x7fbdb0161c80 2024-04-02T16:23:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.046+0000 7fbd97fff640 1 -- 172.21.15.50:0/3060965961 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbda801a070 con 0x7fbdb0161c80 2024-04-02T16:23:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.046+0000 7fbd97fff640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.046+0000 7fbd97fff640 1 -- 172.21.15.50:0/3060965961 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7fbda808b7a0 con 0x7fbdb0161c80 2024-04-02T16:23:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.046+0000 7fbdaf7fe640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.047+0000 7fbdaf7fe640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0073220 tx=0x7fbda0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.051+0000 7fbd97fff640 1 -- 172.21.15.50:0/3060965961 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbda8059d40 con 0x7fbdb0161c80 2024-04-02T16:23:09.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.246+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbdb00ce970 con 0x7fbdb0161c80 2024-04-02T16:23:09.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.852+0000 7fbd97fff640 1 -- 172.21.15.50:0/3060965961 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7fbda8059640 con 0x7fbdb0161c80 2024-04-02T16:23:09.854 INFO:teuthology.orchestra.run.smithi050.stderr: removed all blocklist entries 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbd9c06c8f0 msgr2=0x7fbd9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0073220 tx=0x7fbda0008040 comp rx=0 tx=0).stop 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb0161c80 msgr2=0x7fbdb006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb0161c80 0x7fbdb006c2c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbda800caa0 tx=0x7fbda800cf60 comp rx=0 tx=0).stop 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 shutdown_connections 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbdb0161c80 0x7fbdb006c2c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdb00d2e50 0x7fbdb00cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 --2- 172.21.15.50:0/3060965961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdb00d1200 0x7fbdb0152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.855+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 >> 172.21.15.50:0/3060965961 conn(0x7fbdb009d5e0 msgr2=0x7fbdb015d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.856+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 shutdown_connections 2024-04-02T16:23:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.856+0000 7fbdb5dc8640 1 -- 172.21.15.50:0/3060965961 wait complete. 2024-04-02T16:23:09.871 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-02T16:23:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.978+0000 7f86a4806640 1 Processor -- start 2024-04-02T16:23:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.978+0000 7f86a4806640 1 -- start start 2024-04-02T16:23:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.978+0000 7f86a4806640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f86a00d2f10 0x7f86a00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.978+0000 7f86a4806640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a00ceda0 con 0x7f86a00d0e90 2024-04-02T16:23:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.978+0000 7f86a4806640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a00cf040 con 0x7f86a015e510 2024-04-02T16:23:09.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f86a4806640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f86a00d19c0 con 0x7f86a00d2f10 2024-04-02T16:23:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f8695d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f86a00d2f10 0x7f86a00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f869ed9d640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f86a00d0e90 0x7f86a00d12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43588/0 (socket says 172.21.15.50:43588) 2024-04-02T16:23:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f869ed9d640 1 -- 172.21.15.50:0/554342896 learned_addr learned my addr 172.21.15.50:0/554342896 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f8695d9b640 1 -- 172.21.15.50:0/554342896 >> v1:172.21.15.188:6789/0 conn(0x7f86a015e510 legacy=0x7f86a00d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f8695d9b640 1 -- 172.21.15.50:0/554342896 >> v1:172.21.15.50:6789/0 conn(0x7f86a00d0e90 legacy=0x7f86a00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f8695d9b640 1 -- 172.21.15.50:0/554342896 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a00cf320 con 0x7f86a00d2f10 2024-04-02T16:23:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.979+0000 7f869dd9b640 1 -- 172.21.15.50:0/554342896 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174007744 0 0) 0x7f86a00ceda0 con 0x7f86a00d0e90 2024-04-02T16:23:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f8695d9b640 1 --2- 172.21.15.50:0/554342896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d2f10 0x7f86a00cfe00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86880097a0 tx=0x7f868802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1578ed3f965c6a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f869dd9b640 1 -- 172.21.15.50:0/554342896 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688035020 con 0x7f86a00d2f10 2024-04-02T16:23:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f869dd9b640 1 -- 172.21.15.50:0/554342896 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8688031830 con 0x7f86a00d2f10 2024-04-02T16:23:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f869dd9b640 1 -- 172.21.15.50:0/554342896 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688031c00 con 0x7f86a00d2f10 2024-04-02T16:23:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f86a4806640 1 -- 172.21.15.50:0/554342896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d2f10 msgr2=0x7f86a00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f86a4806640 1 --2- 172.21.15.50:0/554342896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d2f10 0x7f86a00cfe00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86880097a0 tx=0x7f868802b8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f86a4806640 1 -- 172.21.15.50:0/554342896 shutdown_connections 2024-04-02T16:23:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f86a4806640 1 --2- 172.21.15.50:0/554342896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d2f10 0x7f86a00cfe00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.980+0000 7f86a4806640 1 -- 172.21.15.50:0/554342896 >> 172.21.15.50:0/554342896 conn(0x7f86a009d570 msgr2=0x7f86a00d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.981+0000 7f86a4806640 1 -- 172.21.15.50:0/554342896 shutdown_connections 2024-04-02T16:23:09.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.981+0000 7f86a4806640 1 -- 172.21.15.50:0/554342896 wait complete. 2024-04-02T16:23:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.982+0000 7f86a4806640 1 Processor -- start 2024-04-02T16:23:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.982+0000 7f86a4806640 1 -- start start 2024-04-02T16:23:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f86a4806640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d0e90 0x7f86a0153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f86a4806640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86a00d2f10 0x7f86a00c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f86a4806640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a015e510 0x7f86a00c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f86a4806640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86a00b3ad0 con 0x7f86a00d0e90 2024-04-02T16:23:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f86a4806640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86a015e8f0 con 0x7f86a015e510 2024-04-02T16:23:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f86a4806640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f86a00b3c50 con 0x7f86a00d2f10 2024-04-02T16:23:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d0e90 0x7f86a0153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d0e90 0x7f86a0153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35288/0 (socket says 172.21.15.50:35288) 2024-04-02T16:23:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 -- 172.21.15.50:0/3828968661 learned_addr learned my addr 172.21.15.50:0/3828968661 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:09.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f8695d9b640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86a00d2f10 0x7f86a00c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869ed9d640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a015e510 0x7f86a00c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 -- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86a00d2f10 msgr2=0x7f86a00c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86a00d2f10 0x7f86a00c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 -- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a015e510 msgr2=0x7f86a00c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a015e510 0x7f86a00c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:09.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 -- 172.21.15.50:0/3828968661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a00c7eb0 con 0x7f86a00d0e90 2024-04-02T16:23:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f869e59c640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d0e90 0x7f86a0153820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f869000c400 tx=0x7f869000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.983+0000 7f86977fe640 1 -- 172.21.15.50:0/3828968661 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690017070 con 0x7f86a00d0e90 2024-04-02T16:23:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.984+0000 7f86977fe640 1 -- 172.21.15.50:0/3828968661 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86900052d0 con 0x7f86a00d0e90 2024-04-02T16:23:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.984+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a0158ad0 con 0x7f86a00d0e90 2024-04-02T16:23:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.984+0000 7f86977fe640 1 -- 172.21.15.50:0/3828968661 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690003830 con 0x7f86a00d0e90 2024-04-02T16:23:09.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.984+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a0159090 con 0x7f86a00d0e90 2024-04-02T16:23:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.985+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a00756f0 con 0x7f86a00d0e90 2024-04-02T16:23:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.985+0000 7f86977fe640 1 -- 172.21.15.50:0/3828968661 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f869001a070 con 0x7f86a00d0e90 2024-04-02T16:23:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.986+0000 7f86977fe640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f868006c8f0 0x7f868006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.986+0000 7f86977fe640 1 -- 172.21.15.50:0/3828968661 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f869008ac40 con 0x7f86a00d0e90 2024-04-02T16:23:09.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.986+0000 7f8695d9b640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f868006c8f0 0x7f868006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:09.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.987+0000 7f8695d9b640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f868006c8f0 0x7f868006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8688036000 tx=0x7f8688008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:09.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:09.990+0000 7f86977fe640 1 -- 172.21.15.50:0/3828968661 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f86a00c7eb0 con 0x7f86a00d0e90 2024-04-02T16:23:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.245+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f86a00c7600 con 0x7f86a00d0e90 2024-04-02T16:23:10.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.245+0000 7f86977fe640 1 -- 172.21.15.50:0/3828968661 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f86900592c0 con 0x7f86a00d0e90 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "auth_dump": [ 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.a", 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBwMAxm+V4IKRAAfLlvRXFY4am6ultvJitXhA==", 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-04-02T16:23:10.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.b", 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBwMAxmhcJaNRAAk6QDmBeJUzVdEnJ0um5B+w==", 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.c", 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBwMAxmjdFkMBAA43Cl690AwszHSmVuV6K34A==", 2024-04-02T16:23:10.248 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mds.d", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBxMAxmTkIeARAAPBkXSr4GSRcEvRwKJqPccw==", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.249 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.0", 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQByMAxm+ouKJxAAcQqKvQxmC4w0l6fMLRdJSA==", 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.1", 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQB1MAxm0OfbJRAAgeRoJsctcMiE34ZL6w8gew==", 2024-04-02T16:23:10.250 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.2", 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQB4MAxmwHNOJBAAqW6k+GzchpBdqZr/IRSeBA==", 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.3", 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQB7MAxmHAM3IhAAXuRf8UC3Su9y1y5ftgkLmQ==", 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.4", 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCBMAxmC3gaKBAASt1YEuCSsfpokgjPWVWzaQ==", 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.5", 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCEMAxmKQg0JhAATb/qM58zw+QbS+oB63E/bQ==", 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.253 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.6", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCHMAxm6RoVJBAAUmYck7HG8K5l18FjFfiFRw==", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "osd.7", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCKMAxm11rnIRAApt58el/7EFrhuffEjIenWg==", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow profile osd", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile osd", 2024-04-02T16:23:10.254 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.0", 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBxMAxmDpViBRAAI+QEeaTzgk7SzJG7sathbQ==", 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.1", 2024-04-02T16:23:10.255 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBxMAxm+7amDhAACmXb4dph7u9ZFU0LXyDCzQ==", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.2", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBxMAxmO3A7CxAA7rH8JU08Yt6Soho9oaGLtg==", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-04-02T16:23:10.256 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.3", 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBxMAxmuP2HFBAAjiZki3fyRvJtJ4mO6vn+fw==", 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow", 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow r", 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow rw", 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow rwx" 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.admin", 2024-04-02T16:23:10.257 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg==", 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr": "allow *", 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow *", 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-mds", 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCQMAxmkex1GhAAO4E2paE6t+jPptqvFn8UFg==", 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-mds" 2024-04-02T16:23:10.258 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-mgr", 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCQMAxmD/V1GhAAghA0IhntB+gnobRGV6md2w==", 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-osd", 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCQMAxmsvt1GhAAQgIUc6mKUfaGEanTLqUuBQ==", 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-osd" 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rbd", 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCQMAxmEwN2GhAAJuXjmavyZCINPdQDjVay8Q==", 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCQMAxm1Al2GhAAuUoiRClPIFqgU2kMTVXvoA==", 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.260 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "client.bootstrap-rgw", 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQCQMAxm3xB2GhAAENqshv3DAYrntZBqiaPWcQ==", 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.261 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.x", 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBwMAxm8qOcJRAAXFEN1C5WhcHJPlmONrBFPA==", 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "entity": "mgr.y", 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "key": "AQBwMAxmxWS5IRAAfhIqHEPQPduTKpEb/FzWyQ==", 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": { 2024-04-02T16:23:10.262 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": "allow *", 2024-04-02T16:23:10.263 INFO:teuthology.orchestra.run.smithi050.stdout: "mon": "allow profile mgr", 2024-04-02T16:23:10.263 INFO:teuthology.orchestra.run.smithi050.stdout: "osd": "allow *" 2024-04-02T16:23:10.263 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.263 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:10.263 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-04-02T16:23:10.263 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T16:23:10.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f868006c8f0 msgr2=0x7f868006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f868006c8f0 0x7f868006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8688036000 tx=0x7f8688008030 comp rx=0 tx=0).stop 2024-04-02T16:23:10.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d0e90 msgr2=0x7f86a0153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d0e90 0x7f86a0153820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f869000c400 tx=0x7f869000c8c0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 shutdown_connections 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86a015e510 0x7f86a00c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86a00d2f10 0x7f86a00c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f868006c8f0 0x7f868006edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 --2- 172.21.15.50:0/3828968661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86a00d0e90 0x7f86a0153820 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 >> 172.21.15.50:0/3828968661 conn(0x7f86a009d570 msgr2=0x7f86a006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 shutdown_connections 2024-04-02T16:23:10.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.247+0000 7f86a4806640 1 -- 172.21.15.50:0/3828968661 wait complete. 2024-04-02T16:23:10.266 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-02T16:23:10.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.367+0000 7f49f699d640 1 Processor -- start 2024-04-02T16:23:10.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.367+0000 7f49f699d640 1 -- start start 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f699d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49f015f780 0x7f49f015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f699d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f00cf870 con 0x7f49f00d47b0 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f699d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f00d1d10 con 0x7f49f00d1130 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f699d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f49f015fb60 con 0x7f49f015f780 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f519a640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f49f00d47b0 0x7f49f00d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43590/0 (socket says 172.21.15.50:43590) 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f519a640 1 -- 172.21.15.50:0/2461418663 learned_addr learned my addr 172.21.15.50:0/2461418663 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339840358 0 0) 0x7f49e4001110 con 0x7f49f00d47b0 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49d8003520 con 0x7f49f00d47b0 2024-04-02T16:23:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f599b640 1 --2- 172.21.15.50:0/2461418663 >> v2:172.21.15.188:3300/0 conn(0x7f49f015f780 0x7f49f015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.368+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1179558761 0 0) 0x7f49d8003520 con 0x7f49f00d47b0 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 >> v2:172.21.15.188:3300/0 conn(0x7f49f015f780 msgr2=0x7f49f015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 --2- 172.21.15.50:0/2461418663 >> v2:172.21.15.188:3300/0 conn(0x7f49f015f780 0x7f49f015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 >> v1:172.21.15.188:6789/0 conn(0x7f49f00d1130 legacy=0x7f49f00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f00cfaf0 con 0x7f49f00d47b0 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f49e4001840 con 0x7f49f00d47b0 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 >> v1:172.21.15.50:6789/0 conn(0x7f49f00d47b0 legacy=0x7f49f00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49d80039b0 0x7f49d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d8004800 0x7f49d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d80051c0 0x7f49d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49f015fb60 con 0x7f49d80051c0 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49d800c160 con 0x7f49d80039b0 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49d800c360 con 0x7f49d8004800 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f619c640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d80051c0 0x7f49d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f599b640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49d80039b0 0x7f49d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f519a640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d8004800 0x7f49d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f619c640 1 -- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d8004800 msgr2=0x7f49d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f619c640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d8004800 0x7f49d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f619c640 1 -- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49d80039b0 msgr2=0x7f49d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f619c640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49d80039b0 0x7f49d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.369+0000 7f49f619c640 1 -- 172.21.15.50:0/2461418663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d8003520 con 0x7f49d80051c0 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f619c640 1 -- 172.21.15.50:0/2461418663 reap_dead start 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f619c640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d80051c0 0x7f49d800b9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49f0163170 tx=0x7f49d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9c705970b81183 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0001a70 con 0x7f49d80051c0 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0003b70 con 0x7f49d80051c0 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f4999640 1 -- 172.21.15.50:0/2461418663 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e00050d0 con 0x7f49d80051c0 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f699d640 1 -- 172.21.15.50:0/2461418663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d80051c0 msgr2=0x7f49d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f699d640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d80051c0 0x7f49d800b9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49f0163170 tx=0x7f49d8005830 comp rx=0 tx=0).stop 2024-04-02T16:23:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f699d640 1 -- 172.21.15.50:0/2461418663 shutdown_connections 2024-04-02T16:23:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f699d640 1 --2- 172.21.15.50:0/2461418663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49d80051c0 0x7f49d800b9c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.370+0000 7f49f699d640 1 -- 172.21.15.50:0/2461418663 >> 172.21.15.50:0/2461418663 conn(0x7f49f009d410 msgr2=0x7f49f00d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.371+0000 7f49f699d640 1 -- 172.21.15.50:0/2461418663 shutdown_connections 2024-04-02T16:23:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.371+0000 7f49f699d640 1 -- 172.21.15.50:0/2461418663 wait complete. 2024-04-02T16:23:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 Processor -- start 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 -- start start 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d80051c0 0x7f49f00b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49f00cc760 0x7f49f00b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f00ccbe0 0x7f49f00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00d00b0 con 0x7f49f00cc760 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00d2040 con 0x7f49f00ccbe0 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f699d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49f00d4c30 con 0x7f49d80051c0 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.372+0000 7f49f519a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49f00cc760 0x7f49f00b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49f00cc760 0x7f49f00b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35300/0 (socket says 172.21.15.50:35300) 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 -- 172.21.15.50:0/2913984329 learned_addr learned my addr 172.21.15.50:0/2913984329 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:10.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f619c640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f00ccbe0 0x7f49f00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f599b640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d80051c0 0x7f49f00b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 -- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d80051c0 msgr2=0x7f49f00b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d80051c0 0x7f49f00b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 -- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f00ccbe0 msgr2=0x7f49f00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f00ccbe0 0x7f49f00b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 -- 172.21.15.50:0/2913984329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f00b46b0 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f519a640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49f00cc760 0x7f49f00b3a90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49e40072c0 tx=0x7f49e4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49ee7fc640 1 -- 172.21.15.50:0/2913984329 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e401a070 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49ee7fc640 1 -- 172.21.15.50:0/2913984329 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e400f960 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f0159ef0 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49ee7fc640 1 -- 172.21.15.50:0/2913984329 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400fd50 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.373+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f015a4b0 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.374+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f00a1ff0 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.374+0000 7f49ee7fc640 1 -- 172.21.15.50:0/2913984329 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49e401d070 con 0x7f49f00cc760 2024-04-02T16:23:10.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.375+0000 7f49ee7fc640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f49dc06c8f0 0x7f49dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.375+0000 7f49ee7fc640 1 -- 172.21.15.50:0/2913984329 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f49e408b4b0 con 0x7f49f00cc760 2024-04-02T16:23:10.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.375+0000 7f49f599b640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f49dc06c8f0 0x7f49dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.376+0000 7f49f599b640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f49dc06c8f0 0x7f49dc06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009a80 tx=0x7f49e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:10.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.379+0000 7f49ee7fc640 1 -- 172.21.15.50:0/2913984329 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f49f00cf870 con 0x7f49f00cc760 2024-04-02T16:23:10.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.632+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f49f01611e0 con 0x7f49f00cc760 2024-04-02T16:23:10.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.634+0000 7f49ee7fc640 1 -- 172.21.15.50:0/2913984329 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f49e4059aa0 con 0x7f49f00cc760 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f49dc06c8f0 msgr2=0x7f49dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f49dc06c8f0 0x7f49dc06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009a80 tx=0x7f49e8008040 comp rx=0 tx=0).stop 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49f00cc760 msgr2=0x7f49f00b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49f00cc760 0x7f49f00b3a90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49e40072c0 tx=0x7f49e4007780 comp rx=0 tx=0).stop 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 shutdown_connections 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f00ccbe0 0x7f49f00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f49dc06c8f0 0x7f49dc06edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49f00cc760 0x7f49f00b3a90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 --2- 172.21.15.50:0/2913984329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49d80051c0 0x7f49f00b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 >> 172.21.15.50:0/2913984329 conn(0x7f49f009d410 msgr2=0x7f49f00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:10.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 shutdown_connections 2024-04-02T16:23:10.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.636+0000 7f49f699d640 1 -- 172.21.15.50:0/2913984329 wait complete. 2024-04-02T16:23:10.648 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-02T16:23:10.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.750+0000 7f443ae86640 1 Processor -- start 2024-04-02T16:23:10.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.750+0000 7f443ae86640 1 -- start start 2024-04-02T16:23:10.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f443ae86640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f443415e3d0 0x7f44340d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f443ae86640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f44340d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f443ae86640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44340d2f30 con 0x7f44340c7e30 2024-04-02T16:23:10.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f443ae86640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f44340cf000 con 0x7f443415e3d0 2024-04-02T16:23:10.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f443ae86640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44340cf2e0 con 0x7f443415e7b0 2024-04-02T16:23:10.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f4438c22640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f443415e3d0 0x7f44340d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f442bfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f44340c7e30 0x7f44340d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43600/0 (socket says 172.21.15.50:43600) 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f442bfff640 1 -- 172.21.15.50:0/2297796851 learned_addr learned my addr 172.21.15.50:0/2297796851 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f4439423640 1 --2- 172.21.15.50:0/2297796851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f44340d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f4438c22640 1 -- 172.21.15.50:0/2297796851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 msgr2=0x7f44340d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f4438c22640 1 --2- 172.21.15.50:0/2297796851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f44340d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f4438c22640 1 -- 172.21.15.50:0/2297796851 >> v1:172.21.15.50:6789/0 conn(0x7f44340c7e30 legacy=0x7f44340d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f4438c22640 1 -- 172.21.15.50:0/2297796851 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44340cf5c0 con 0x7f443415e3d0 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f442b7fe640 1 -- 172.21.15.50:0/2297796851 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701830433 0 0) 0x7f44340d2f30 con 0x7f44340c7e30 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.751+0000 7f4439423640 1 --2- 172.21.15.50:0/2297796851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f44340d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.752+0000 7f4438c22640 1 --2- 172.21.15.50:0/2297796851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f44340d4030 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f44300097a0 tx=0x7f443002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5fd4fc13b4486a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.752+0000 7f442b7fe640 1 -- 172.21.15.50:0/2297796851 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430035020 con 0x7f443415e3d0 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.752+0000 7f442b7fe640 1 -- 172.21.15.50:0/2297796851 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4430031830 con 0x7f443415e3d0 2024-04-02T16:23:10.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.752+0000 7f442b7fe640 1 -- 172.21.15.50:0/2297796851 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430031c00 con 0x7f443415e3d0 2024-04-02T16:23:10.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 -- 172.21.15.50:0/2297796851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 msgr2=0x7f44340d4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 --2- 172.21.15.50:0/2297796851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f44340d4030 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f44300097a0 tx=0x7f443002b8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 -- 172.21.15.50:0/2297796851 shutdown_connections 2024-04-02T16:23:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 --2- 172.21.15.50:0/2297796851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f44340d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 --2- 172.21.15.50:0/2297796851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f44340d4030 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 -- 172.21.15.50:0/2297796851 >> 172.21.15.50:0/2297796851 conn(0x7f443408dc90 msgr2=0x7f443409d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 -- 172.21.15.50:0/2297796851 shutdown_connections 2024-04-02T16:23:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.753+0000 7f443ae86640 1 -- 172.21.15.50:0/2297796851 wait complete. 2024-04-02T16:23:10.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.754+0000 7f443ae86640 1 Processor -- start 2024-04-02T16:23:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.754+0000 7f443ae86640 1 -- start start 2024-04-02T16:23:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f443ae86640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44340c7e30 0x7f4434157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f443ae86640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f443415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f443ae86640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f443415af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f443ae86640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44340d46f0 con 0x7f443415e3d0 2024-04-02T16:23:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f443ae86640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44340d4570 con 0x7f44340c7e30 2024-04-02T16:23:10.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f443ae86640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44340d4870 con 0x7f443415e7b0 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f443415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f443415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35314/0 (socket says 172.21.15.50:35314) 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 -- 172.21.15.50:0/821486660 learned_addr learned my addr 172.21.15.50:0/821486660 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f4439423640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f443415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 -- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 msgr2=0x7f443415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f443415af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 -- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44340c7e30 msgr2=0x7f4434157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44340c7e30 0x7f4434157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 -- 172.21.15.50:0/821486660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44340b9290 con 0x7f443415e3d0 2024-04-02T16:23:10.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f442bfff640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f443415a9f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f442400c410 tx=0x7f442400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f44297fa640 1 -- 172.21.15.50:0/821486660 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424017070 con 0x7f443415e3d0 2024-04-02T16:23:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f44297fa640 1 -- 172.21.15.50:0/821486660 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4424005ce0 con 0x7f443415e3d0 2024-04-02T16:23:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.755+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44340b9570 con 0x7f443415e3d0 2024-04-02T16:23:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.756+0000 7f44297fa640 1 -- 172.21.15.50:0/821486660 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240072f0 con 0x7f443415e3d0 2024-04-02T16:23:10.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.756+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44340b9b30 con 0x7f443415e3d0 2024-04-02T16:23:10.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.757+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434075760 con 0x7f443415e3d0 2024-04-02T16:23:10.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.757+0000 7f44297fa640 1 -- 172.21.15.50:0/821486660 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f442401a070 con 0x7f443415e3d0 2024-04-02T16:23:10.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.757+0000 7f44297fa640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f442006c8f0 0x7f442006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:10.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.757+0000 7f44297fa640 1 -- 172.21.15.50:0/821486660 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f442408ad90 con 0x7f443415e3d0 2024-04-02T16:23:10.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.757+0000 7f4438c22640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f442006c8f0 0x7f442006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:10.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.758+0000 7f4438c22640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f442006c8f0 0x7f442006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4430036000 tx=0x7f4430030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:10.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:10.762+0000 7f44297fa640 1 -- 172.21.15.50:0/821486660 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f44240592f0 con 0x7f443415e3d0 2024-04-02T16:23:11.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.019+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f44340d51d0 con 0x7f443415e3d0 2024-04-02T16:23:11.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.021+0000 7f44297fa640 1 -- 172.21.15.50:0/821486660 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f4424058bf0 con 0x7f443415e3d0 2024-04-02T16:23:11.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f442006c8f0 msgr2=0x7f442006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f442006c8f0 0x7f442006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4430036000 tx=0x7f4430030000 comp rx=0 tx=0).stop 2024-04-02T16:23:11.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 msgr2=0x7f443415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f443415a9f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f442400c410 tx=0x7f442400c8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 shutdown_connections 2024-04-02T16:23:11.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443415e7b0 0x7f443415af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f442006c8f0 0x7f442006edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f443415e3d0 0x7f443415a9f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 --2- 172.21.15.50:0/821486660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44340c7e30 0x7f4434157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 >> 172.21.15.50:0/821486660 conn(0x7f443408dc90 msgr2=0x7f443406d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:11.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 shutdown_connections 2024-04-02T16:23:11.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.023+0000 7f443ae86640 1 -- 172.21.15.50:0/821486660 wait complete. 2024-04-02T16:23:11.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-02T16:23:11.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.136+0000 7f7884455640 1 Processor -- start 2024-04-02T16:23:11.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.136+0000 7f7884455640 1 -- start start 2024-04-02T16:23:11.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.137+0000 7f7884455640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f787c15e7b0 0x7f787c0d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.137+0000 7f7884455640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787c0d2f30 con 0x7f787c0c7e30 2024-04-02T16:23:11.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.137+0000 7f7884455640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787c0cf000 con 0x7f787c15e3d0 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.137+0000 7f7884455640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f787c0cf2e0 con 0x7f787c15e7b0 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.137+0000 7f78819f0640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f787c0c7e30 0x7f787c0d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43602/0 (socket says 172.21.15.50:43602) 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.137+0000 7f78819f0640 1 -- 172.21.15.50:0/908364171 learned_addr learned my addr 172.21.15.50:0/908364171 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78829f2640 1 --2- 172.21.15.50:0/908364171 >> v2:172.21.15.188:3300/0 conn(0x7f787c15e7b0 0x7f787c0d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844711229 0 0) 0x7f787c0d2f30 con 0x7f787c0c7e30 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7864003520 con 0x7f787c0c7e30 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1413028796 0 0) 0x7f7864003520 con 0x7f787c0c7e30 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 >> v2:172.21.15.188:3300/0 conn(0x7f787c15e7b0 msgr2=0x7f787c0d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 --2- 172.21.15.50:0/908364171 >> v2:172.21.15.188:3300/0 conn(0x7f787c15e7b0 0x7f787c0d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 >> v1:172.21.15.188:6789/0 conn(0x7f787c15e3d0 legacy=0x7f787c0d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c0cf5c0 con 0x7f787c0c7e30 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f7870001670 con 0x7f787c0c7e30 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 >> v1:172.21.15.50:6789/0 conn(0x7f787c0c7e30 legacy=0x7f787c0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78640039b0 0x7f7864003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78640043b0 0x7f7864004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7864005080 0x7f786400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0cf2e0 con 0x7f7864005080 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f786400bfe0 con 0x7f78640043b0 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f786400c1e0 con 0x7f78640039b0 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78829f2640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7864005080 0x7f786400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78821f1640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78640039b0 0x7f7864003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78819f0640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78640043b0 0x7f7864004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78829f2640 1 -- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78640039b0 msgr2=0x7f7864003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78829f2640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78640039b0 0x7f7864003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78829f2640 1 -- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78640043b0 msgr2=0x7f7864004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78829f2640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78640043b0 0x7f7864004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.138+0000 7f78829f2640 1 -- 172.21.15.50:0/908364171 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7864003520 con 0x7f7864005080 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.139+0000 7f78829f2640 1 -- 172.21.15.50:0/908364171 reap_dead start 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.139+0000 7f78829f2640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7864005080 0x7f786400b9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7874009ac0 tx=0x7f7864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aa7bf2b4b8ab5d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.139+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874009690 con 0x7f7864005080 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.139+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7874004020 con 0x7f7864005080 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.139+0000 7f78811ef640 1 -- 172.21.15.50:0/908364171 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740043d0 con 0x7f7864005080 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.140+0000 7f7884455640 1 -- 172.21.15.50:0/908364171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7864005080 msgr2=0x7f786400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.140+0000 7f7884455640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7864005080 0x7f786400b9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7874009ac0 tx=0x7f7864005830 comp rx=0 tx=0).stop 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.140+0000 7f7884455640 1 -- 172.21.15.50:0/908364171 shutdown_connections 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.140+0000 7f7884455640 1 --2- 172.21.15.50:0/908364171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7864005080 0x7f786400b9c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.140+0000 7f7884455640 1 -- 172.21.15.50:0/908364171 >> 172.21.15.50:0/908364171 conn(0x7f787c08dc90 msgr2=0x7f787c09d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:11.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.140+0000 7f7884455640 1 -- 172.21.15.50:0/908364171 shutdown_connections 2024-04-02T16:23:11.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.140+0000 7f7884455640 1 -- 172.21.15.50:0/908364171 wait complete. 2024-04-02T16:23:11.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.141+0000 7f7884455640 1 Processor -- start 2024-04-02T16:23:11.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.141+0000 7f7884455640 1 -- start start 2024-04-02T16:23:11.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f7884455640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7864005080 0x7f787c0cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f7884455640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c09f920 0x7f787c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f7884455640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f787c09fda0 0x7f787c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f7884455640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0d41b0 con 0x7f787c09f920 2024-04-02T16:23:11.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f7884455640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0d4030 con 0x7f7864005080 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f7884455640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f787c0d4330 con 0x7f787c09fda0 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c09f920 0x7f787c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c09f920 0x7f787c06c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35328/0 (socket says 172.21.15.50:35328) 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 -- 172.21.15.50:0/1213859395 learned_addr learned my addr 172.21.15.50:0/1213859395 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 -- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f787c09fda0 msgr2=0x7f787c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f787c09fda0 0x7f787c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 -- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7864005080 msgr2=0x7f787c0cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7864005080 0x7f787c0cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 -- 172.21.15.50:0/1213859395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c1f4450 con 0x7f787c09f920 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.142+0000 7f78819f0640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c09f920 0x7f787c06c2a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f787000e780 tx=0x7f787000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.143+0000 7f787affd640 1 -- 172.21.15.50:0/1213859395 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000c870 con 0x7f787c09f920 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.143+0000 7f787affd640 1 -- 172.21.15.50:0/1213859395 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7870018070 con 0x7f787c09f920 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.143+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c1f4730 con 0x7f787c09f920 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.143+0000 7f787affd640 1 -- 172.21.15.50:0/1213859395 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000cc50 con 0x7f787c09f920 2024-04-02T16:23:11.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.143+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1f4cf0 con 0x7f787c09f920 2024-04-02T16:23:11.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.144+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c075760 con 0x7f787c09f920 2024-04-02T16:23:11.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.144+0000 7f787affd640 1 -- 172.21.15.50:0/1213859395 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f787001d460 con 0x7f787c09f920 2024-04-02T16:23:11.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.145+0000 7f787affd640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f786806c8f0 0x7f786806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.145+0000 7f787affd640 1 -- 172.21.15.50:0/1213859395 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f787008b8c0 con 0x7f787c09f920 2024-04-02T16:23:11.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.145+0000 7f78821f1640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f786806c8f0 0x7f786806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.145+0000 7f78821f1640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f786806c8f0 0x7f786806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f787c0c7c30 tx=0x7f786c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.149+0000 7f787affd640 1 -- 172.21.15.50:0/1213859395 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7870059eb0 con 0x7f787c09f920 2024-04-02T16:23:11.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.405+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f787c0d51d0 con 0x7f787c09f920 2024-04-02T16:23:11.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.407+0000 7f787affd640 1 -- 172.21.15.50:0/1213859395 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f78700597b0 con 0x7f787c09f920 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f786806c8f0 msgr2=0x7f786806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f786806c8f0 0x7f786806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f787c0c7c30 tx=0x7f786c009040 comp rx=0 tx=0).stop 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c09f920 msgr2=0x7f787c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c09f920 0x7f787c06c2a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f787000e780 tx=0x7f787000ec40 comp rx=0 tx=0).stop 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 shutdown_connections 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f787c09fda0 0x7f787c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f786806c8f0 0x7f786806edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c09f920 0x7f787c06c2a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 --2- 172.21.15.50:0/1213859395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7864005080 0x7f787c0cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 >> 172.21.15.50:0/1213859395 conn(0x7f787c08dc90 msgr2=0x7f787c15c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 shutdown_connections 2024-04-02T16:23:11.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.409+0000 7f7884455640 1 -- 172.21.15.50:0/1213859395 wait complete. 2024-04-02T16:23:11.422 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-02T16:23:11.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.524+0000 7f161ede9640 1 Processor -- start 2024-04-02T16:23:11.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.524+0000 7f161ede9640 1 -- start start 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f161ede9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161815fb20 0x7f1618070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f161ede9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16180ce660 0x7f161809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f161ede9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16180cd810 con 0x7f161809af80 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f161ede9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f161809b9d0 con 0x7f16180ce660 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f161ede9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f161815ff00 con 0x7f161815fb20 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f1617fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16180ce660 0x7f161809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f161cb85640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161815fb20 0x7f1618070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f1617fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16180ce660 0x7f161809aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.50:52258/0 (socket says 172.21.15.50:52258) 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.525+0000 7f1617fff640 1 -- 172.21.15.50:0/2754735018 learned_addr learned my addr 172.21.15.50:0/2754735018 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.526+0000 7f1617fff640 1 -- 172.21.15.50:0/2754735018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161815fb20 msgr2=0x7f1618070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.526+0000 7f1617fff640 1 --2- 172.21.15.50:0/2754735018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161815fb20 0x7f1618070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.526+0000 7f1617fff640 1 -- 172.21.15.50:0/2754735018 >> v1:172.21.15.188:6789/0 conn(0x7f161809af80 legacy=0x7f16180cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.526+0000 7f1617fff640 1 -- 172.21.15.50:0/2754735018 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161809b6f0 con 0x7f16180ce660 2024-04-02T16:23:11.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.526+0000 7f16177fe640 1 -- 172.21.15.50:0/2754735018 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330353018 0 0) 0x7f16180cd810 con 0x7f161809af80 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.526+0000 7f1617fff640 1 --2- 172.21.15.50:0/2754735018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16180ce660 0x7f161809aa40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1608009e50 tx=0x7f160802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2166a91d30375ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f16177fe640 1 -- 172.21.15.50:0/2754735018 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160802cb00 con 0x7f16180ce660 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f16177fe640 1 -- 172.21.15.50:0/2754735018 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160802cca0 con 0x7f16180ce660 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f16177fe640 1 -- 172.21.15.50:0/2754735018 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608034610 con 0x7f16180ce660 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 -- 172.21.15.50:0/2754735018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16180ce660 msgr2=0x7f161809aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 --2- 172.21.15.50:0/2754735018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16180ce660 0x7f161809aa40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1608009e50 tx=0x7f160802bf60 comp rx=0 tx=0).stop 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 -- 172.21.15.50:0/2754735018 shutdown_connections 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 --2- 172.21.15.50:0/2754735018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16180ce660 0x7f161809aa40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 --2- 172.21.15.50:0/2754735018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161815fb20 0x7f1618070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 -- 172.21.15.50:0/2754735018 >> 172.21.15.50:0/2754735018 conn(0x7f16180b3be0 msgr2=0x7f16180b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 -- 172.21.15.50:0/2754735018 shutdown_connections 2024-04-02T16:23:11.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.527+0000 7f161ede9640 1 -- 172.21.15.50:0/2754735018 wait complete. 2024-04-02T16:23:11.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.528+0000 7f161ede9640 1 Processor -- start 2024-04-02T16:23:11.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.528+0000 7f161ede9640 1 -- start start 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161ede9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161809af80 0x7f16181f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161ede9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16180ce660 0x7f16180c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161ede9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161815fb20 0x7f16180c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161ede9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f16180cdd90 con 0x7f16180ce660 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161ede9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16180d1000 con 0x7f161815fb20 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161ede9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f161809b410 con 0x7f161809af80 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f1617fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16180ce660 0x7f16180c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f1617fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16180ce660 0x7f16180c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35338/0 (socket says 172.21.15.50:35338) 2024-04-02T16:23:11.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f1617fff640 1 -- 172.21.15.50:0/701411136 learned_addr learned my addr 172.21.15.50:0/701411136 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161d386640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161815fb20 0x7f16180c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.529+0000 7f161cb85640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161809af80 0x7f16181f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f1617fff640 1 -- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161809af80 msgr2=0x7f16181f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f1617fff640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161809af80 0x7f16181f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f1617fff640 1 -- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161815fb20 msgr2=0x7f16180c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f1617fff640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161815fb20 0x7f16180c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f1617fff640 1 -- 172.21.15.50:0/701411136 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161806c6d0 con 0x7f16180ce660 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f1617fff640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16180ce660 0x7f16180c91c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1608036040 tx=0x7f1608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f16157fa640 1 -- 172.21.15.50:0/701411136 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608043020 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f16157fa640 1 -- 172.21.15.50:0/701411136 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160802ce50 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161806c9b0 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f16157fa640 1 -- 172.21.15.50:0/701411136 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608009510 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.530+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16180b6b20 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.531+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1618075760 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.531+0000 7f16157fa640 1 -- 172.21.15.50:0/701411136 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1608046020 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.532+0000 7f16157fa640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f15f806c8f0 0x7f15f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.532+0000 7f16157fa640 1 -- 172.21.15.50:0/701411136 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f16080adec0 con 0x7f16180ce660 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.532+0000 7f161cb85640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f15f806c8f0 0x7f15f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.533+0000 7f161cb85640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f15f806c8f0 0x7f15f806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1610009b40 tx=0x7f1610009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.537+0000 7f16157fa640 1 -- 172.21.15.50:0/701411136 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f160804d030 con 0x7f16180ce660 2024-04-02T16:23:11.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.790+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f16180b4fb0 con 0x7f16180ce660 2024-04-02T16:23:11.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.792+0000 7f16157fa640 1 -- 172.21.15.50:0/701411136 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f160807bef0 con 0x7f16180ce660 2024-04-02T16:23:11.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f15f806c8f0 msgr2=0x7f15f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f15f806c8f0 0x7f15f806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1610009b40 tx=0x7f1610009190 comp rx=0 tx=0).stop 2024-04-02T16:23:11.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16180ce660 msgr2=0x7f16180c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16180ce660 0x7f16180c91c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1608036040 tx=0x7f1608004a40 comp rx=0 tx=0).stop 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 shutdown_connections 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161815fb20 0x7f16180c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f15f806c8f0 0x7f15f806edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16180ce660 0x7f16180c91c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 --2- 172.21.15.50:0/701411136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161809af80 0x7f16181f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 >> 172.21.15.50:0/701411136 conn(0x7f16180b3be0 msgr2=0x7f16180c7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 shutdown_connections 2024-04-02T16:23:11.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.794+0000 7f161ede9640 1 -- 172.21.15.50:0/701411136 wait complete. 2024-04-02T16:23:11.806 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-02T16:23:11.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.909+0000 7fb9cc19d640 1 Processor -- start 2024-04-02T16:23:11.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9cc19d640 1 -- start start 2024-04-02T16:23:11.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9cc19d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 0x7fb9c40d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9cc19d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb9c40d0850 0x7fb9c40d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9cc19d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9c40d4b10 con 0x7fb9c415e5d0 2024-04-02T16:23:11.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9cc19d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb9c415e9b0 con 0x7fb9c40d0850 2024-04-02T16:23:11.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9cc19d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c40d03c0 con 0x7fb9c40d4170 2024-04-02T16:23:11.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9c9f39640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fb9c415e5d0 0x7fb9c40cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43616/0 (socket says 172.21.15.50:43616) 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.910+0000 7fb9c9f39640 1 -- 172.21.15.50:0/2497269085 learned_addr learned my addr 172.21.15.50:0/2497269085 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9738640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 0x7fb9c40d45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9ca73a640 1 --2- 172.21.15.50:0/2497269085 >> v2:172.21.15.188:3300/0 conn(0x7fb9c40d0850 0x7fb9c40d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042722785 0 0) 0x7fb9c40d4b10 con 0x7fb9c415e5d0 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9b0003520 con 0x7fb9c415e5d0 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 80396999 0 0) 0x7fb9b0003520 con 0x7fb9c415e5d0 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 msgr2=0x7fb9c40d45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 0x7fb9c40d45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 >> v2:172.21.15.188:3300/0 conn(0x7fb9c40d0850 msgr2=0x7fb9c40d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 --2- 172.21.15.50:0/2497269085 >> v2:172.21.15.188:3300/0 conn(0x7fb9c40d0850 0x7fb9c40d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c40cf7b0 con 0x7fb9c415e5d0 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fb9c0001670 con 0x7fb9c415e5d0 2024-04-02T16:23:11.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 >> v1:172.21.15.50:6789/0 conn(0x7fb9c415e5d0 legacy=0x7fb9c40cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9b00039b0 0x7fb9b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b0004800 0x7fb9b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 0x7fb9b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c415e9b0 con 0x7fb9b00039b0 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40d03c0 con 0x7fb9b0004800 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b000b9c0 con 0x7fb9c40d4170 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9b00039b0 0x7fb9b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9738640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b0004800 0x7fb9b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9ca73a640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 0x7fb9b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9f39640 1 -- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 msgr2=0x7fb9b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40d4170 0x7fb9b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9f39640 1 -- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b0004800 msgr2=0x7fb9b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b0004800 0x7fb9b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.911+0000 7fb9c9f39640 1 -- 172.21.15.50:0/2497269085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b0003520 con 0x7fb9b00039b0 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.912+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9b00039b0 0x7fb9b0003dd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001050 tx=0x7fb9b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747641792db197ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.912+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0007d70 con 0x7fb9b00039b0 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.912+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c0030070 con 0x7fb9b00039b0 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.912+0000 7fb9c8f37640 1 -- 172.21.15.50:0/2497269085 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c002cd80 con 0x7fb9b00039b0 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.913+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2497269085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9b00039b0 msgr2=0x7fb9b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.913+0000 7fb9cc19d640 1 --2- 172.21.15.50:0/2497269085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9b00039b0 0x7fb9b0003dd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001050 tx=0x7fb9b0005830 comp rx=0 tx=0).stop 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.913+0000 7fb9ca73a640 1 -- 172.21.15.50:0/2497269085 reap_dead start 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.913+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2497269085 shutdown_connections 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.913+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2497269085 >> 172.21.15.50:0/2497269085 conn(0x7fb9c408dce0 msgr2=0x7fb9c409d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.913+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2497269085 shutdown_connections 2024-04-02T16:23:11.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.913+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2497269085 wait complete. 2024-04-02T16:23:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.914+0000 7fb9cc19d640 1 Processor -- start 2024-04-02T16:23:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.914+0000 7fb9cc19d640 1 -- start start 2024-04-02T16:23:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9cc19d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b00039b0 0x7fb9c40b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9cc19d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b34d0 0x7fb9c41f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9cc19d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40b89e0 0x7fb9c40b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9cc19d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40d0050 con 0x7fb9c40b34d0 2024-04-02T16:23:11.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9cc19d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40cfd70 con 0x7fb9b00039b0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9cc19d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c40cf3e0 con 0x7fb9c40b89e0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b34d0 0x7fb9c41f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b34d0 0x7fb9c41f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35346/0 (socket says 172.21.15.50:35346) 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 -- 172.21.15.50:0/2148871593 learned_addr learned my addr 172.21.15.50:0/2148871593 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9ca73a640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40b89e0 0x7fb9c40b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b00039b0 0x7fb9c40b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 -- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40b89e0 msgr2=0x7fb9c40b8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40b89e0 0x7fb9c40b8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 -- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b00039b0 msgr2=0x7fb9c40b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b00039b0 0x7fb9c40b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 -- 172.21.15.50:0/2148871593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c40b74c0 con 0x7fb9c40b34d0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b00039b0 0x7fb9c40b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.915+0000 7fb9c9738640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b34d0 0x7fb9c41f6f10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40118e0 tx=0x7fb9b4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.916+0000 7fb9ba7fc640 1 -- 172.21.15.50:0/2148871593 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400faf0 con 0x7fb9c40b34d0 2024-04-02T16:23:11.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.916+0000 7fb9ba7fc640 1 -- 172.21.15.50:0/2148871593 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b400fc90 con 0x7fb9c40b34d0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.916+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c40b77a0 con 0x7fb9c40b34d0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.916+0000 7fb9ba7fc640 1 -- 172.21.15.50:0/2148871593 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4014440 con 0x7fb9c40b34d0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.916+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c40b7d60 con 0x7fb9c40b34d0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.917+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c4075760 con 0x7fb9c40b34d0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.917+0000 7fb9ba7fc640 1 -- 172.21.15.50:0/2148871593 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9b4012040 con 0x7fb9c40b34d0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.917+0000 7fb9ba7fc640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9a806c8f0 0x7fb9a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.918+0000 7fb9ba7fc640 1 -- 172.21.15.50:0/2148871593 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fb9b408bf70 con 0x7fb9c40b34d0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.918+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9a806c8f0 0x7fb9a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:11.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.918+0000 7fb9c9f39640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9a806c8f0 0x7fb9a806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0034000 tx=0x7fb9c0007000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:11.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:11.922+0000 7fb9ba7fc640 1 -- 172.21.15.50:0/2148871593 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb9b4019030 con 0x7fb9c40b34d0 2024-04-02T16:23:12.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.177+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fb9c408e3b0 con 0x7fb9c40b34d0 2024-04-02T16:23:12.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.180+0000 7fb9ba7fc640 1 -- 172.21.15.50:0/2148871593 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fb9b405a5f0 con 0x7fb9c40b34d0 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9a806c8f0 msgr2=0x7fb9a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9a806c8f0 0x7fb9a806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0034000 tx=0x7fb9c0007000 comp rx=0 tx=0).stop 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b34d0 msgr2=0x7fb9c41f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b34d0 0x7fb9c41f6f10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40118e0 tx=0x7fb9b4011da0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 shutdown_connections 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9c40b89e0 0x7fb9c40b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fb9a806c8f0 0x7fb9a806edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9c40b34d0 0x7fb9c41f6f10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 --2- 172.21.15.50:0/2148871593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9b00039b0 0x7fb9c40b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 >> 172.21.15.50:0/2148871593 conn(0x7fb9c408dce0 msgr2=0x7fb9c406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 shutdown_connections 2024-04-02T16:23:12.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.181+0000 7fb9cc19d640 1 -- 172.21.15.50:0/2148871593 wait complete. 2024-04-02T16:23:12.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-02T16:23:12.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.294+0000 7fe4f8da1640 1 Processor -- start 2024-04-02T16:23:12.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.294+0000 7fe4f8da1640 1 -- start start 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f8da1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4f40d09d0 0x7fe4f415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f8da1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe4f40d0db0 0x7fe4f4161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f8da1640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f40cf2f0 con 0x7fe4f4162490 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f8da1640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fe4f415f9a0 con 0x7fe4f40d0db0 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f8da1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe4f40cf5d0 con 0x7fe4f40d09d0 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f2ffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe4f40d0db0 0x7fe4f4161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f3fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe4f4162490 0x7fe4f40d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43630/0 (socket says 172.21.15.50:43630) 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f3fff640 1 -- 172.21.15.50:0/1930024551 learned_addr learned my addr 172.21.15.50:0/1930024551 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:12.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f2ffd640 1 -- 172.21.15.50:0/1930024551 >> v2:172.21.15.188:3300/0 conn(0x7fe4f40d09d0 msgr2=0x7fe4f415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:12.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f2ffd640 1 --2- 172.21.15.50:0/1930024551 >> v2:172.21.15.188:3300/0 conn(0x7fe4f40d09d0 0x7fe4f415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f2ffd640 1 -- 172.21.15.50:0/1930024551 >> v1:172.21.15.50:6789/0 conn(0x7fe4f4162490 legacy=0x7fe4f40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.295+0000 7fe4f2ffd640 1 -- 172.21.15.50:0/1930024551 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f40cf8b0 con 0x7fe4f40d0db0 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.296+0000 7fe4f2ffd640 1 --2- 172.21.15.50:0/1930024551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d0db0 0x7fe4f4161f50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40cb1f0 tx=0x7fe4e802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdb3f1da8b01c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.296+0000 7fe4f27fc640 1 -- 172.21.15.50:0/1930024551 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8034070 con 0x7fe4f40d0db0 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.296+0000 7fe4f27fc640 1 -- 172.21.15.50:0/1930024551 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e8030610 con 0x7fe4f40d0db0 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.296+0000 7fe4f27fc640 1 -- 172.21.15.50:0/1930024551 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80309e0 con 0x7fe4f40d0db0 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.296+0000 7fe4f8da1640 1 -- 172.21.15.50:0/1930024551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d0db0 msgr2=0x7fe4f4161f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.296+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/1930024551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d0db0 0x7fe4f4161f50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40cb1f0 tx=0x7fe4e802b960 comp rx=0 tx=0).stop 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.296+0000 7fe4f8da1640 1 -- 172.21.15.50:0/1930024551 shutdown_connections 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.297+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/1930024551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d0db0 0x7fe4f4161f50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.297+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/1930024551 >> v2:172.21.15.188:3300/0 conn(0x7fe4f40d09d0 0x7fe4f415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.297+0000 7fe4f8da1640 1 -- 172.21.15.50:0/1930024551 >> 172.21.15.50:0/1930024551 conn(0x7fe4f409d490 msgr2=0x7fe4f40d4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:12.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.297+0000 7fe4f8da1640 1 -- 172.21.15.50:0/1930024551 shutdown_connections 2024-04-02T16:23:12.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.297+0000 7fe4f8da1640 1 -- 172.21.15.50:0/1930024551 wait complete. 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.298+0000 7fe4f8da1640 1 Processor -- start 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.298+0000 7fe4f8da1640 1 -- start start 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.298+0000 7fe4f8da1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d09d0 0x7fe4f40cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.298+0000 7fe4f8da1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f40d0db0 0x7fe4f4159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.298+0000 7fe4f8da1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4162490 0x7fe4f40ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.298+0000 7fe4f8da1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f40d1730 con 0x7fe4f40d09d0 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.298+0000 7fe4f8da1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f40cfe70 con 0x7fe4f4162490 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f8da1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f40d11f0 con 0x7fe4f40d0db0 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d09d0 0x7fe4f40cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d09d0 0x7fe4f40cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35374/0 (socket says 172.21.15.50:35374) 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 -- 172.21.15.50:0/314877190 learned_addr learned my addr 172.21.15.50:0/314877190 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:12.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f2ffd640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f40d0db0 0x7fe4f4159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f3fff640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4162490 0x7fe4f40ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 -- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f40d0db0 msgr2=0x7fe4f4159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f40d0db0 0x7fe4f4159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 -- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4162490 msgr2=0x7fe4f40ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4162490 0x7fe4f40ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 -- 172.21.15.50:0/314877190 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f40b9aa0 con 0x7fe4f40d09d0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f37fe640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d09d0 0x7fe4f40cb240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b440 tx=0x7fe4e400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4dbfff640 1 -- 172.21.15.50:0/314877190 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4015020 con 0x7fe4f40d09d0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4dbfff640 1 -- 172.21.15.50:0/314877190 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e4011890 con 0x7fe4f40d09d0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4dbfff640 1 -- 172.21.15.50:0/314877190 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4011c80 con 0x7fe4f40d09d0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f4161dd0 con 0x7fe4f40d09d0 2024-04-02T16:23:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.299+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f40b9700 con 0x7fe4f40d09d0 2024-04-02T16:23:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.300+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f404f940 con 0x7fe4f40d09d0 2024-04-02T16:23:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.301+0000 7fe4dbfff640 1 -- 172.21.15.50:0/314877190 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4e4018020 con 0x7fe4f40d09d0 2024-04-02T16:23:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.301+0000 7fe4dbfff640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe4e006c920 0x7fe4e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.301+0000 7fe4dbfff640 1 -- 172.21.15.50:0/314877190 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fe4e408b9e0 con 0x7fe4f40d09d0 2024-04-02T16:23:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.302+0000 7fe4f2ffd640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe4e006c920 0x7fe4e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.302+0000 7fe4f2ffd640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe4e006c920 0x7fe4e006ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8036000 tx=0x7fe4e803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:12.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.306+0000 7fe4dbfff640 1 -- 172.21.15.50:0/314877190 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe4e401f020 con 0x7fe4f40d09d0 2024-04-02T16:23:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.566+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fe4f408e240 con 0x7fe4f40d09d0 2024-04-02T16:23:12.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.568+0000 7fe4dbfff640 1 -- 172.21.15.50:0/314877190 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fe4e400d040 con 0x7fe4f40d09d0 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe4e006c920 msgr2=0x7fe4e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe4e006c920 0x7fe4e006ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8036000 tx=0x7fe4e803b040 comp rx=0 tx=0).stop 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d09d0 msgr2=0x7fe4f40cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d09d0 0x7fe4f40cb240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b440 tx=0x7fe4e400b900 comp rx=0 tx=0).stop 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 shutdown_connections 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f4162490 0x7fe4f40ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f40d0db0 0x7fe4f4159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe4e006c920 0x7fe4e006ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 --2- 172.21.15.50:0/314877190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f40d09d0 0x7fe4f40cb240 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 >> 172.21.15.50:0/314877190 conn(0x7fe4f409d490 msgr2=0x7fe4f40b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 shutdown_connections 2024-04-02T16:23:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.570+0000 7fe4f8da1640 1 -- 172.21.15.50:0/314877190 wait complete. 2024-04-02T16:23:12.582 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:23:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.682+0000 7f291bf50640 1 Processor -- start 2024-04-02T16:23:12.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.682+0000 7f291bf50640 1 -- start start 2024-04-02T16:23:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f291bf50640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f29141414e0 0x7f291415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f291bf50640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f291415f040 0x7f29140b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f291bf50640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29140b3ca0 0x7f2914143d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f291bf50640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f291415f510 con 0x7f29141414e0 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f291bf50640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f291415f7f0 con 0x7f291415f040 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f291bf50640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f291406b6c0 con 0x7f29140b3ca0 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f29141414e0 0x7f291415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f29141414e0 0x7f291415eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35380/0 (socket says 172.21.15.50:35380) 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 -- 172.21.15.50:0/2956480914 learned_addr learned my addr 172.21.15.50:0/2956480914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f291a4ed640 1 --2- 172.21.15.50:0/2956480914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29140b3ca0 0x7f2914143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f29194eb640 1 --2- 172.21.15.50:0/2956480914 >> v2:172.21.15.188:3300/0 conn(0x7f291415f040 0x7f29140b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 -- 172.21.15.50:0/2956480914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29140b3ca0 msgr2=0x7f2914143d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 --2- 172.21.15.50:0/2956480914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29140b3ca0 0x7f2914143d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 -- 172.21.15.50:0/2956480914 >> v2:172.21.15.188:3300/0 conn(0x7f291415f040 msgr2=0x7f29140b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 --2- 172.21.15.50:0/2956480914 >> v2:172.21.15.188:3300/0 conn(0x7f291415f040 0x7f29140b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f2919cec640 1 -- 172.21.15.50:0/2956480914 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29141418c0 con 0x7f29141414e0 2024-04-02T16:23:12.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.683+0000 7f29194eb640 1 --2- 172.21.15.50:0/2956480914 >> v2:172.21.15.188:3300/0 conn(0x7f291415f040 0x7f29140b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.684+0000 7f2919cec640 1 --2- 172.21.15.50:0/2956480914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29141414e0 0x7f291415eb00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f29140b5e50 tx=0x7f291002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8038ec81317caf5b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.684+0000 7f2918cea640 1 -- 172.21.15.50:0/2956480914 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910035020 con 0x7f29141414e0 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.684+0000 7f2918cea640 1 -- 172.21.15.50:0/2956480914 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2910031870 con 0x7f29141414e0 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f2918cea640 1 -- 172.21.15.50:0/2956480914 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910031c40 con 0x7f29141414e0 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 -- 172.21.15.50:0/2956480914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29141414e0 msgr2=0x7f291415eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 --2- 172.21.15.50:0/2956480914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29141414e0 0x7f291415eb00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f29140b5e50 tx=0x7f291002b910 comp rx=0 tx=0).stop 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 -- 172.21.15.50:0/2956480914 shutdown_connections 2024-04-02T16:23:12.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 --2- 172.21.15.50:0/2956480914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29140b3ca0 0x7f2914143d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 --2- 172.21.15.50:0/2956480914 >> v2:172.21.15.188:3300/0 conn(0x7f291415f040 0x7f29140b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 --2- 172.21.15.50:0/2956480914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29141414e0 0x7f291415eb00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 -- 172.21.15.50:0/2956480914 >> 172.21.15.50:0/2956480914 conn(0x7f29140d0790 msgr2=0x7f29140d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 -- 172.21.15.50:0/2956480914 shutdown_connections 2024-04-02T16:23:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.685+0000 7f291bf50640 1 -- 172.21.15.50:0/2956480914 wait complete. 2024-04-02T16:23:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.686+0000 7f291bf50640 1 Processor -- start 2024-04-02T16:23:12.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.686+0000 7f291bf50640 1 -- start start 2024-04-02T16:23:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291bf50640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29140b3ca0 0x7f2914140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291bf50640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29141414e0 0x7f29140ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291bf50640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f291415f040 0x7f29140cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291bf50640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f29140b4560 con 0x7f291415f040 2024-04-02T16:23:12.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291bf50640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29140b43e0 con 0x7f29140b3ca0 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291bf50640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f29141442a0 con 0x7f29141414e0 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f291415f040 0x7f29140cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f291415f040 0x7f29140cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35382/0 (socket says 172.21.15.50:35382) 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 -- 172.21.15.50:0/2689415087 learned_addr learned my addr 172.21.15.50:0/2689415087 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f29194eb640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29141414e0 0x7f29140ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f2919cec640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29140b3ca0 0x7f2914140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 -- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29141414e0 msgr2=0x7f29140ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29141414e0 0x7f29140ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 -- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29140b3ca0 msgr2=0x7f2914140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29140b3ca0 0x7f2914140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f291a4ed640 1 -- 172.21.15.50:0/2689415087 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29141e4970 con 0x7f291415f040 2024-04-02T16:23:12.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.687+0000 7f29194eb640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29141414e0 0x7f29140ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:12.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.688+0000 7f291a4ed640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f291415f040 0x7f29140cec00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f290c00eed0 tx=0x7f290c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:12.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.688+0000 7f290a7fc640 1 -- 172.21.15.50:0/2689415087 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c018020 con 0x7f291415f040 2024-04-02T16:23:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.688+0000 7f290a7fc640 1 -- 172.21.15.50:0/2689415087 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c014400 con 0x7f291415f040 2024-04-02T16:23:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.688+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29141e4c50 con 0x7f291415f040 2024-04-02T16:23:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.688+0000 7f290a7fc640 1 -- 172.21.15.50:0/2689415087 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0147f0 con 0x7f291415f040 2024-04-02T16:23:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.688+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29141e5210 con 0x7f291415f040 2024-04-02T16:23:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.689+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29140756f0 con 0x7f291415f040 2024-04-02T16:23:12.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.689+0000 7f290a7fc640 1 -- 172.21.15.50:0/2689415087 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f290c005d10 con 0x7f291415f040 2024-04-02T16:23:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.689+0000 7f290a7fc640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f28f806c920 0x7f28f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.690+0000 7f290a7fc640 1 -- 172.21.15.50:0/2689415087 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f290c08bf80 con 0x7f291415f040 2024-04-02T16:23:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.690+0000 7f2919cec640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f28f806c920 0x7f28f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.690+0000 7f2919cec640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f28f806c920 0x7f28f806ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2910036000 tx=0x7f291003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:12.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.694+0000 7f290a7fc640 1 -- 172.21.15.50:0/2689415087 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f290c022070 con 0x7f291415f040 2024-04-02T16:23:12.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.877+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29140b0f40 con 0x7f291415f040 2024-04-02T16:23:12.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.878+0000 7f290a7fc640 1 -- 172.21.15.50:0/2689415087 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13152 (secure 0 0 0) 0x7f290c05a600 con 0x7f291415f040 2024-04-02T16:23:12.879 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:12.879 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":35,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:23:09.860645+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:23:12.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f28f806c920 msgr2=0x7f28f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f28f806c920 0x7f28f806ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2910036000 tx=0x7f291003c040 comp rx=0 tx=0).stop 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f291415f040 msgr2=0x7f29140cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f291415f040 0x7f29140cec00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f290c00eed0 tx=0x7f290c00b480 comp rx=0 tx=0).stop 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 shutdown_connections 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f28f806c920 0x7f28f806ede0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f291415f040 0x7f29140cec00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29141414e0 0x7f29140ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 --2- 172.21.15.50:0/2689415087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29140b3ca0 0x7f2914140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 >> 172.21.15.50:0/2689415087 conn(0x7f29140d0790 msgr2=0x7f29140b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 shutdown_connections 2024-04-02T16:23:12.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.880+0000 7f291bf50640 1 -- 172.21.15.50:0/2689415087 wait complete. 2024-04-02T16:23:12.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:23:12.892 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:23:12.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.996+0000 7faff1022640 1 Processor -- start 2024-04-02T16:23:12.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.996+0000 7faff1022640 1 -- start start 2024-04-02T16:23:13.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.996+0000 7faff1022640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fafec161ca0 0x7fafec0d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.996+0000 7faff1022640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 0x7fafec0d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.996+0000 7faff1022640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafec163040 con 0x7fafec162bb0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.996+0000 7faff1022640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fafec0d15f0 con 0x7fafec161ca0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.996+0000 7faff1022640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafec0d5000 con 0x7fafec0d0c80 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafea59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fafec162bb0 0x7fafec0cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43634/0 (socket says 172.21.15.50:43634) 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafea59c640 1 -- 172.21.15.50:0/2525342694 learned_addr learned my addr 172.21.15.50:0/2525342694 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafead9d640 1 --2- 172.21.15.50:0/2525342694 >> v2:172.21.15.188:3300/0 conn(0x7fafec161ca0 0x7fafec0d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafeb59e640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 0x7fafec0d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599328590 0 0) 0x7fafec163040 con 0x7fafec162bb0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafd4003520 con 0x7fafec162bb0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 421264991 0 0) 0x7fafd4003520 con 0x7fafec162bb0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 msgr2=0x7fafec0d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 0x7fafec0d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 >> v2:172.21.15.188:3300/0 conn(0x7fafec161ca0 msgr2=0x7fafec0d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 --2- 172.21.15.50:0/2525342694 >> v2:172.21.15.188:3300/0 conn(0x7fafec161ca0 0x7fafec0d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec0cf0d0 con 0x7fafec162bb0 2024-04-02T16:23:13.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fafe0001670 con 0x7fafec162bb0 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafeb59e640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 0x7fafec0d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafead9d640 1 --2- 172.21.15.50:0/2525342694 >> v2:172.21.15.188:3300/0 conn(0x7fafec161ca0 0x7fafec0d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 >> v1:172.21.15.50:6789/0 conn(0x7fafec162bb0 legacy=0x7fafec0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafd40039b0 0x7fafd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd4004800 0x7fafd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 0x7fafd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0d15f0 con 0x7fafd40039b0 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0d5000 con 0x7fafd4004800 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafd400ba10 con 0x7fafec0d0c80 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafead9d640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafd40039b0 0x7fafd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafea59c640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd4004800 0x7fafd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafeb59e640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 0x7fafd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.997+0000 7fafead9d640 1 -- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 msgr2=0x7fafd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:13.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafead9d640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafec0d0c80 0x7fafd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafead9d640 1 -- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd4004800 msgr2=0x7fafd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafead9d640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd4004800 0x7fafd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafead9d640 1 -- 172.21.15.50:0/2525342694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd4003520 con 0x7fafd40039b0 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafea59c640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd4004800 0x7fafd4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafead9d640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafd40039b0 0x7fafd4003dd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009c90 tx=0x7fafd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a7270143497c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc039070 con 0x7fafd40039b0 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc02c980 con 0x7fafd40039b0 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.998+0000 7fafe9d9b640 1 -- 172.21.15.50:0/2525342694 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc009860 con 0x7fafd40039b0 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.999+0000 7faff1022640 1 -- 172.21.15.50:0/2525342694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafd40039b0 msgr2=0x7fafd4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.999+0000 7faff1022640 1 --2- 172.21.15.50:0/2525342694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafd40039b0 0x7fafd4003dd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009c90 tx=0x7fafd4005830 comp rx=0 tx=0).stop 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.999+0000 7fafeb59e640 1 -- 172.21.15.50:0/2525342694 reap_dead start 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.999+0000 7faff1022640 1 -- 172.21.15.50:0/2525342694 shutdown_connections 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.999+0000 7faff1022640 1 -- 172.21.15.50:0/2525342694 >> 172.21.15.50:0/2525342694 conn(0x7fafec09d5c0 msgr2=0x7fafec0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.999+0000 7faff1022640 1 -- 172.21.15.50:0/2525342694 shutdown_connections 2024-04-02T16:23:13.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:12.999+0000 7faff1022640 1 -- 172.21.15.50:0/2525342694 wait complete. 2024-04-02T16:23:13.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.000+0000 7faff1022640 1 Processor -- start 2024-04-02T16:23:13.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.000+0000 7faff1022640 1 -- start start 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7faff1022640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40039b0 0x7fafec0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7faff1022640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0ca3e0 0x7fafec149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7faff1022640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec0ca840 0x7fafec0cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7faff1022640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0ced80 con 0x7fafec0ca3e0 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7faff1022640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0cf690 con 0x7fafec0ca840 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7faff1022640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafec0d10b0 con 0x7fafd40039b0 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7fafea59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0ca3e0 0x7fafec149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7fafea59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0ca3e0 0x7fafec149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35410/0 (socket says 172.21.15.50:35410) 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.001+0000 7fafea59c640 1 -- 172.21.15.50:0/2455899306 learned_addr learned my addr 172.21.15.50:0/2455899306 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafead9d640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40039b0 0x7fafec0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafeb59e640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec0ca840 0x7fafec0cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafea59c640 1 -- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40039b0 msgr2=0x7fafec0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafea59c640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40039b0 0x7fafec0c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafea59c640 1 -- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec0ca840 msgr2=0x7fafec0cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafea59c640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec0ca840 0x7fafec0cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafea59c640 1 -- 172.21.15.50:0/2455899306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec0c9940 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafead9d640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40039b0 0x7fafec0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafea59c640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0ca3e0 0x7fafec149a60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fafe0001200 tx=0x7fafe000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafd37fe640 1 -- 172.21.15.50:0/2455899306 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000c660 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafd37fe640 1 -- 172.21.15.50:0/2455899306 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe0018070 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7fafd37fe640 1 -- 172.21.15.50:0/2455899306 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000ca20 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafec14a480 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.002+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafec0c9be0 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.003+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafec075760 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.004+0000 7fafd37fe640 1 -- 172.21.15.50:0/2455899306 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafe001d460 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.004+0000 7fafd37fe640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.004+0000 7fafd37fe640 1 -- 172.21.15.50:0/2455899306 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fafe008ba40 con 0x7fafec0ca3e0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.004+0000 7fafead9d640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.005+0000 7fafead9d640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafd806c8f0 0x7fafd806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafdc036000 tx=0x7fafdc030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.008+0000 7fafd37fe640 1 -- 172.21.15.50:0/2455899306 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fafe0059fa0 con 0x7fafec0ca3e0 2024-04-02T16:23:13.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.193+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafec0d0a60 con 0x7fafec0ca3e0 2024-04-02T16:23:13.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.193+0000 7fafd37fe640 1 -- 172.21.15.50:0/2455899306 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fafe008be20 con 0x7fafec0ca3e0 2024-04-02T16:23:13.194 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T16:23:13.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafd806c8f0 msgr2=0x7fafd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafd806c8f0 0x7fafd806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafdc036000 tx=0x7fafdc030000 comp rx=0 tx=0).stop 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0ca3e0 msgr2=0x7fafec149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0ca3e0 0x7fafec149a60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fafe0001200 tx=0x7fafe000eb90 comp rx=0 tx=0).stop 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 shutdown_connections 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafec0ca840 0x7fafec0cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafec0ca3e0 0x7fafec149a60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 --2- 172.21.15.50:0/2455899306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40039b0 0x7fafec0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 >> 172.21.15.50:0/2455899306 conn(0x7fafec09d5c0 msgr2=0x7fafec06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 shutdown_connections 2024-04-02T16:23:13.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.195+0000 7faff1022640 1 -- 172.21.15.50:0/2455899306 wait complete. 2024-04-02T16:23:13.207 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T16:23:13.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 Processor -- start 2024-04-02T16:23:13.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 -- start start 2024-04-02T16:23:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa32015e670 0x7fa3200b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32006cff0 0x7fa3200d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa32009aa40 0x7fa32009aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fa32006da20 con 0x7fa32015e670 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa3200d0b00 con 0x7fa32009aa40 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa3275e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa3200d0210 con 0x7fa32006cff0 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa325385640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa32015e670 0x7fa3200b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa325385640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fa32015e670 0x7fa3200b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35426/0 (socket says 172.21.15.50:35426) 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.308+0000 7fa325385640 1 -- 172.21.15.50:0/785089024 learned_addr learned my addr 172.21.15.50:0/785089024 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa324b84640 1 --2- 172.21.15.50:0/785089024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32006cff0 0x7fa3200d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325b86640 1 --2- 172.21.15.50:0/785089024 >> v2:172.21.15.188:3300/0 conn(0x7fa32009aa40 0x7fa32009aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325385640 1 -- 172.21.15.50:0/785089024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32006cff0 msgr2=0x7fa3200d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325385640 1 --2- 172.21.15.50:0/785089024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32006cff0 0x7fa3200d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325385640 1 -- 172.21.15.50:0/785089024 >> v2:172.21.15.188:3300/0 conn(0x7fa32009aa40 msgr2=0x7fa32009aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325385640 1 --2- 172.21.15.50:0/785089024 >> v2:172.21.15.188:3300/0 conn(0x7fa32009aa40 0x7fa32009aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325385640 1 -- 172.21.15.50:0/785089024 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32006d740 con 0x7fa32015e670 2024-04-02T16:23:13.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325b86640 1 --2- 172.21.15.50:0/785089024 >> v2:172.21.15.188:3300/0 conn(0x7fa32009aa40 0x7fa32009aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.309+0000 7fa325385640 1 --2- 172.21.15.50:0/785089024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32015e670 0x7fa3200b4680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa320073160 tx=0x7fa31802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d6bca9489e510f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa317fff640 1 -- 172.21.15.50:0/785089024 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318034070 con 0x7fa32015e670 2024-04-02T16:23:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa317fff640 1 -- 172.21.15.50:0/785089024 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa318030860 con 0x7fa32015e670 2024-04-02T16:23:13.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa317fff640 1 -- 172.21.15.50:0/785089024 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318030c30 con 0x7fa32015e670 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 -- 172.21.15.50:0/785089024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32015e670 msgr2=0x7fa3200b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 --2- 172.21.15.50:0/785089024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32015e670 0x7fa3200b4680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa320073160 tx=0x7fa31802b9a0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 -- 172.21.15.50:0/785089024 shutdown_connections 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 --2- 172.21.15.50:0/785089024 >> v2:172.21.15.188:3300/0 conn(0x7fa32009aa40 0x7fa32009aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 --2- 172.21.15.50:0/785089024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32006cff0 0x7fa3200d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 --2- 172.21.15.50:0/785089024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32015e670 0x7fa3200b4680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 -- 172.21.15.50:0/785089024 >> 172.21.15.50:0/785089024 conn(0x7fa3200b3b90 msgr2=0x7fa3200b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 -- 172.21.15.50:0/785089024 shutdown_connections 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.310+0000 7fa3275e9640 1 -- 172.21.15.50:0/785089024 wait complete. 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.311+0000 7fa3275e9640 1 Processor -- start 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.311+0000 7fa3275e9640 1 -- start start 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa3275e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32006cff0 0x7fa3201573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa3275e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32009aa40 0x7fa320157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa3275e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa32015e670 0x7fa3201ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa3275e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa32006d450 con 0x7fa32006cff0 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa3275e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa3200d0790 con 0x7fa32015e670 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa3275e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa3200d14d0 con 0x7fa32009aa40 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa325385640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32006cff0 0x7fa3201573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa325385640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32006cff0 0x7fa3201573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35438/0 (socket says 172.21.15.50:35438) 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa325385640 1 -- 172.21.15.50:0/2386764534 learned_addr learned my addr 172.21.15.50:0/2386764534 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa325b86640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa32015e670 0x7fa3201ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.312+0000 7fa324b84640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32009aa40 0x7fa320157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa325385640 1 -- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32009aa40 msgr2=0x7fa320157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa325385640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32009aa40 0x7fa320157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa325385640 1 -- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa32015e670 msgr2=0x7fa3201ea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa325385640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa32015e670 0x7fa3201ea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa325385640 1 -- 172.21.15.50:0/2386764534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3201578e0 con 0x7fa32006cff0 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa325b86640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa32015e670 0x7fa3201ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:13.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa325385640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32006cff0 0x7fa3201573a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa318000c00 tx=0x7fa3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa315ffb640 1 -- 172.21.15.50:0/2386764534 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318034040 con 0x7fa32006cff0 2024-04-02T16:23:13.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa315ffb640 1 -- 172.21.15.50:0/2386764534 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3180433f0 con 0x7fa32006cff0 2024-04-02T16:23:13.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa320158480 con 0x7fa32006cff0 2024-04-02T16:23:13.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa315ffb640 1 -- 172.21.15.50:0/2386764534 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3180437e0 con 0x7fa32006cff0 2024-04-02T16:23:13.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.313+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3201e9530 con 0x7fa32006cff0 2024-04-02T16:23:13.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.314+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa320068e80 con 0x7fa32006cff0 2024-04-02T16:23:13.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.315+0000 7fa315ffb640 1 -- 172.21.15.50:0/2386764534 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa318037070 con 0x7fa32006cff0 2024-04-02T16:23:13.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.315+0000 7fa315ffb640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.315+0000 7fa315ffb640 1 -- 172.21.15.50:0/2386764534 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa3180adb90 con 0x7fa32006cff0 2024-04-02T16:23:13.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.315+0000 7fa324b84640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.316+0000 7fa324b84640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa30406c8f0 0x7fa30406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa31000aaf0 tx=0x7fa310008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.320+0000 7fa315ffb640 1 -- 172.21.15.50:0/2386764534 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa320068e80 con 0x7fa32006cff0 2024-04-02T16:23:13.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.508+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa3200b8020 con 0x7fa32006cff0 2024-04-02T16:23:13.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.509+0000 7fa315ffb640 1 -- 172.21.15.50:0/2386764534 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7fa31803e070 con 0x7fa32006cff0 2024-04-02T16:23:13.509 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:13.509 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-04-02T16:23:13.509 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-04-02T16:23:13.509 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-04-02T16:23:13.509 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-04-02T16:23:13.510 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T16:23:13.510 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa30406c8f0 msgr2=0x7fa30406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa30406c8f0 0x7fa30406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa31000aaf0 tx=0x7fa310008040 comp rx=0 tx=0).stop 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32006cff0 msgr2=0x7fa3201573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32006cff0 0x7fa3201573a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa318000c00 tx=0x7fa3180046c0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 shutdown_connections 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa32015e670 0x7fa3201ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa32009aa40 0x7fa320157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 --2- 172.21.15.50:0/2386764534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa32006cff0 0x7fa3201573a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 >> 172.21.15.50:0/2386764534 conn(0x7fa3200b3b90 msgr2=0x7fa3200b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 shutdown_connections 2024-04-02T16:23:13.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.511+0000 7fa3275e9640 1 -- 172.21.15.50:0/2386764534 wait complete. 2024-04-02T16:23:13.521 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T16:23:13.522 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T16:23:13.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.627+0000 7f34e9c47640 1 Processor -- start 2024-04-02T16:23:13.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.627+0000 7f34e9c47640 1 -- start start 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.628+0000 7f34e9c47640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f34e4162570 0x7f34e40cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.628+0000 7f34e9c47640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e40d0670 con 0x7f34e4162950 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.628+0000 7f34e9c47640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e40d0950 con 0x7f34e40d0110 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.628+0000 7f34e9c47640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f34e40d0c30 con 0x7f34e4162570 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.628+0000 7f34e37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f34e4162570 0x7f34e40cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.628+0000 7f34e3fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f34e4162950 0x7f34e40c9f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43636/0 (socket says 172.21.15.50:43636) 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.628+0000 7f34e3fff640 1 -- 172.21.15.50:0/1187105065 learned_addr learned my addr 172.21.15.50:0/1187105065 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.629+0000 7f34e37fe640 1 -- 172.21.15.50:0/1187105065 >> v1:172.21.15.188:6789/0 conn(0x7f34e40d0110 legacy=0x7f34e415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.629+0000 7f34e37fe640 1 -- 172.21.15.50:0/1187105065 >> v1:172.21.15.50:6789/0 conn(0x7f34e4162950 legacy=0x7f34e40c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.629+0000 7f34e37fe640 1 -- 172.21.15.50:0/1187105065 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e40d0f10 con 0x7f34e4162570 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.629+0000 7f34e37fe640 1 --2- 172.21.15.50:0/1187105065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e4162570 0x7f34e40cfbd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f34d40097a0 tx=0x7f34d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e671d280dbb8ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.629+0000 7f34e27fc640 1 -- 172.21.15.50:0/1187105065 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4035020 con 0x7f34e4162570 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.629+0000 7f34e27fc640 1 -- 172.21.15.50:0/1187105065 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d4031830 con 0x7f34e4162570 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.629+0000 7f34e27fc640 1 -- 172.21.15.50:0/1187105065 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4031c00 con 0x7f34e4162570 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.630+0000 7f34e9c47640 1 -- 172.21.15.50:0/1187105065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e4162570 msgr2=0x7f34e40cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.630+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1187105065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e4162570 0x7f34e40cfbd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f34d40097a0 tx=0x7f34d402b8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.630+0000 7f34e9c47640 1 -- 172.21.15.50:0/1187105065 shutdown_connections 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.630+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1187105065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e4162570 0x7f34e40cfbd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.630+0000 7f34e9c47640 1 -- 172.21.15.50:0/1187105065 >> 172.21.15.50:0/1187105065 conn(0x7f34e409d5a0 msgr2=0x7f34e40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.630+0000 7f34e9c47640 1 -- 172.21.15.50:0/1187105065 shutdown_connections 2024-04-02T16:23:13.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.630+0000 7f34e9c47640 1 -- 172.21.15.50:0/1187105065 wait complete. 2024-04-02T16:23:13.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.631+0000 7f34e9c47640 1 Processor -- start 2024-04-02T16:23:13.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.631+0000 7f34e9c47640 1 -- start start 2024-04-02T16:23:13.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e9c47640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e40d0110 0x7f34e4155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e9c47640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34e4162570 0x7f34e41f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e9c47640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34e4162950 0x7f34e41f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e9c47640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4163160 con 0x7f34e40d0110 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e9c47640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4162fe0 con 0x7f34e4162950 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e9c47640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34e40d1710 con 0x7f34e4162570 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e40d0110 0x7f34e4155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e40d0110 0x7f34e4155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35444/0 (socket says 172.21.15.50:35444) 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 -- 172.21.15.50:0/1160084315 learned_addr learned my addr 172.21.15.50:0/1160084315 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 -- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34e4162570 msgr2=0x7f34e41f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34e4162570 0x7f34e41f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 -- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34e4162950 msgr2=0x7f34e41f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34e4162950 0x7f34e41f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.632+0000 7f34e37fe640 1 -- 172.21.15.50:0/1160084315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e4159d50 con 0x7f34e40d0110 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.633+0000 7f34e37fe640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e40d0110 0x7f34e4155ea0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f34d40098c0 tx=0x7f34d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.633+0000 7f34cbfff640 1 -- 172.21.15.50:0/1160084315 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d403ba80 con 0x7f34e40d0110 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.633+0000 7f34cbfff640 1 -- 172.21.15.50:0/1160084315 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d403bc40 con 0x7f34e40d0110 2024-04-02T16:23:13.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.633+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e40c8940 con 0x7f34e40d0110 2024-04-02T16:23:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.633+0000 7f34cbfff640 1 -- 172.21.15.50:0/1160084315 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40453f0 con 0x7f34e40d0110 2024-04-02T16:23:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.633+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e40c91e0 con 0x7f34e40d0110 2024-04-02T16:23:13.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.634+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e40c8b10 con 0x7f34e40d0110 2024-04-02T16:23:13.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.634+0000 7f34cbfff640 1 -- 172.21.15.50:0/1160084315 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34d404c060 con 0x7f34e40d0110 2024-04-02T16:23:13.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.635+0000 7f34cbfff640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34d006c8f0 0x7f34d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:13.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.635+0000 7f34cbfff640 1 -- 172.21.15.50:0/1160084315 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f34d40ad6f0 con 0x7f34e40d0110 2024-04-02T16:23:13.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.635+0000 7f34e2ffd640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34d006c8f0 0x7f34d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:13.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.636+0000 7f34e2ffd640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34d006c8f0 0x7f34d006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f34e415a770 tx=0x7f34d8006ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:13.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.639+0000 7f34cbfff640 1 -- 172.21.15.50:0/1160084315 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f34d4040070 con 0x7f34e40d0110 2024-04-02T16:23:13.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:13.823+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f34e40757d0 con 0x7f34e40d0110 2024-04-02T16:23:14.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.572+0000 7f34cbfff640 1 -- 172.21.15.50:0/1160084315 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f34d407bd70 con 0x7f34e40d0110 2024-04-02T16:23:14.573 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-04-02T16:23:14.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.576+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34d006c8f0 msgr2=0x7f34d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:14.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.576+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34d006c8f0 0x7f34d006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f34e415a770 tx=0x7f34d8006ba0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.576+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e40d0110 msgr2=0x7f34e4155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:14.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.576+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e40d0110 0x7f34e4155ea0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f34d40098c0 tx=0x7f34d40046c0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.577+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 shutdown_connections 2024-04-02T16:23:14.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.577+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34e4162950 0x7f34e41f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.577+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34e4162570 0x7f34e41f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.577+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f34d006c8f0 0x7f34d006edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.577+0000 7f34e9c47640 1 --2- 172.21.15.50:0/1160084315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34e40d0110 0x7f34e4155ea0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.577+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 >> 172.21.15.50:0/1160084315 conn(0x7f34e409d5a0 msgr2=0x7f34e40b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:14.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.577+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 shutdown_connections 2024-04-02T16:23:14.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.578+0000 7f34e9c47640 1 -- 172.21.15.50:0/1160084315 wait complete. 2024-04-02T16:23:14.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T16:23:14.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.709+0000 7f3fac262640 1 Processor -- start 2024-04-02T16:23:14.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.709+0000 7f3fac262640 1 -- start start 2024-04-02T16:23:14.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fac262640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fa4140fc0 0x7f3fa40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:14.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fac262640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa4141690 con 0x7f3fa413e0b0 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fac262640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa40cf3d0 con 0x7f3fa40cef40 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fac262640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa4071360 con 0x7f3fa4140fc0 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fa97fd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fa4140fc0 0x7f3fa40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fa9ffe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f3fa413e0b0 0x7f3fa41408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:43650/0 (socket says 172.21.15.50:43650) 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fa9ffe640 1 -- 172.21.15.50:0/2100066158 learned_addr learned my addr 172.21.15.50:0/2100066158 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fa97fd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3fa4140fc0 0x7f3fa40cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35454/0 (socket says 172.21.15.50:35454) 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fa97fd640 1 -- 172.21.15.50:0/2100066158 >> v1:172.21.15.188:6789/0 conn(0x7f3fa40cef40 legacy=0x7f3fa4071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fa97fd640 1 -- 172.21.15.50:0/2100066158 >> v1:172.21.15.50:6789/0 conn(0x7f3fa413e0b0 legacy=0x7f3fa41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:14.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.710+0000 7f3fa97fd640 1 -- 172.21.15.50:0/2100066158 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa40cf670 con 0x7f3fa4140fc0 2024-04-02T16:23:14.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.711+0000 7f3fa97fd640 1 --2- 172.21.15.50:0/2100066158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa40cea00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40714e0 tx=0x7f3f9402b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bb535eafd01cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:14.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.711+0000 7f3fa8ffc640 1 -- 172.21.15.50:0/2100066158 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037070 con 0x7f3fa4140fc0 2024-04-02T16:23:14.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.711+0000 7f3fa8ffc640 1 -- 172.21.15.50:0/2100066158 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94030710 con 0x7f3fa4140fc0 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.711+0000 7f3fa8ffc640 1 -- 172.21.15.50:0/2100066158 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94030ae0 con 0x7f3fa4140fc0 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.712+0000 7f3fac262640 1 -- 172.21.15.50:0/2100066158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 msgr2=0x7f3fa40cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.712+0000 7f3fac262640 1 --2- 172.21.15.50:0/2100066158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa40cea00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40714e0 tx=0x7f3f9402b6f0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.712+0000 7f3fac262640 1 -- 172.21.15.50:0/2100066158 shutdown_connections 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.712+0000 7f3fac262640 1 --2- 172.21.15.50:0/2100066158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa40cea00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.712+0000 7f3fac262640 1 -- 172.21.15.50:0/2100066158 >> 172.21.15.50:0/2100066158 conn(0x7f3fa406eb90 msgr2=0x7f3fa406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.712+0000 7f3fac262640 1 -- 172.21.15.50:0/2100066158 shutdown_connections 2024-04-02T16:23:14.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.712+0000 7f3fac262640 1 -- 172.21.15.50:0/2100066158 wait complete. 2024-04-02T16:23:14.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.713+0000 7f3fac262640 1 Processor -- start 2024-04-02T16:23:14.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.713+0000 7f3fac262640 1 -- start start 2024-04-02T16:23:14.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fac262640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fa40cef40 0x7f3fa41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:14.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fac262640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fa413e0b0 0x7f3fa41f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:14.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fac262640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa413fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:14.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fac262640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40715b0 con 0x7f3fa4140fc0 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fac262640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40cfc30 con 0x7f3fa413e0b0 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fac262640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa413e490 con 0x7f3fa40cef40 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa413fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa413fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35464/0 (socket says 172.21.15.50:35464) 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 -- 172.21.15.50:0/2985909237 learned_addr learned my addr 172.21.15.50:0/2985909237 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fa9ffe640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fa40cef40 0x7f3fa41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fa97fd640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fa413e0b0 0x7f3fa41f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 -- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fa40cef40 msgr2=0x7f3fa41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fa40cef40 0x7f3fa41589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 -- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fa413e0b0 msgr2=0x7f3fa41f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fa413e0b0 0x7f3fa41f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 -- 172.21.15.50:0/2985909237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa41534f0 con 0x7f3fa4140fc0 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fa9ffe640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fa40cef40 0x7f3fa41589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3fa97fd640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fa413e0b0 0x7f3fa41f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:14.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.714+0000 7f3faa7ff640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa413fca0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00c780 tx=0x7f3f9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:14.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.715+0000 7f3f9a7fc640 1 -- 172.21.15.50:0/2985909237 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c004b60 con 0x7f3fa4140fc0 2024-04-02T16:23:14.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.715+0000 7f3f9a7fc640 1 -- 172.21.15.50:0/2985909237 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9c004d00 con 0x7f3fa4140fc0 2024-04-02T16:23:14.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.715+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa41537d0 con 0x7f3fa4140fc0 2024-04-02T16:23:14.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.715+0000 7f3f9a7fc640 1 -- 172.21.15.50:0/2985909237 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00aa90 con 0x7f3fa4140fc0 2024-04-02T16:23:14.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.715+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa4153d90 con 0x7f3fa4140fc0 2024-04-02T16:23:14.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.716+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa4075760 con 0x7f3fa4140fc0 2024-04-02T16:23:14.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.716+0000 7f3f9a7fc640 1 -- 172.21.15.50:0/2985909237 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f9c00ac30 con 0x7f3fa4140fc0 2024-04-02T16:23:14.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.717+0000 7f3f9a7fc640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f8806c920 0x7f3f8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:14.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.717+0000 7f3f9a7fc640 1 -- 172.21.15.50:0/2985909237 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f3f9c08bcf0 con 0x7f3fa4140fc0 2024-04-02T16:23:14.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.717+0000 7f3fa9ffe640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f8806c920 0x7f3f8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:14.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.718+0000 7f3fa9ffe640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f8806c920 0x7f3f8806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0000f60 tx=0x7f3fa0007d20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:14.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.721+0000 7f3f9a7fc640 1 -- 172.21.15.50:0/2985909237 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3f9c090070 con 0x7f3fa4140fc0 2024-04-02T16:23:14.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:14.907+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3fa40734d0 con 0x7f3fa4140fc0 2024-04-02T16:23:15.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.577+0000 7f3f9a7fc640 1 -- 172.21.15.50:0/2985909237 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f3f9c020070 con 0x7f3fa4140fc0 2024-04-02T16:23:15.578 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.580+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f8806c920 msgr2=0x7f3f8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.580+0000 7f3fac262640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f8806c920 0x7f3f8806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0000f60 tx=0x7f3fa0007d20 comp rx=0 tx=0).stop 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 msgr2=0x7f3fa413fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa413fca0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00c780 tx=0x7f3f9c00cc40 comp rx=0 tx=0).stop 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 shutdown_connections 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f8806c920 0x7f3f8806ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3fa4140fc0 0x7f3fa413fca0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fa413e0b0 0x7f3fa41f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 --2- 172.21.15.50:0/2985909237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fa40cef40 0x7f3fa41589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 >> 172.21.15.50:0/2985909237 conn(0x7f3fa406eb90 msgr2=0x7f3fa406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 shutdown_connections 2024-04-02T16:23:15.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.581+0000 7f3fac262640 1 -- 172.21.15.50:0/2985909237 wait complete. 2024-04-02T16:23:15.610 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T16:23:15.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.720+0000 7f3f5aba8640 1 Processor -- start 2024-04-02T16:23:15.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f5aba8640 1 -- start start 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f5aba8640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3f54161c80 0x7f3f540d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f5aba8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3f540d13d0 0x7f3f540b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f5aba8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f540b10e0 con 0x7f3f540d2e50 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f5aba8640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f3f540b1380 con 0x7f3f54161c80 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f5aba8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3f540b1660 con 0x7f3f540d13d0 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f58944640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3f54161c80 0x7f3f540d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f58944640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f3f54161c80 0x7f3f540d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40886/0 (socket says 172.21.15.50:40886) 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.721+0000 7f3f58944640 1 -- 172.21.15.50:0/3909148501 learned_addr learned my addr 172.21.15.50:0/3909148501 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.722+0000 7f3f59145640 1 --2- 172.21.15.50:0/3909148501 >> v2:172.21.15.188:3300/0 conn(0x7f3f540d13d0 0x7f3f540b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:15.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.722+0000 7f3f58944640 1 -- 172.21.15.50:0/3909148501 >> v2:172.21.15.188:3300/0 conn(0x7f3f540d13d0 msgr2=0x7f3f540b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:15.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.722+0000 7f3f58944640 1 --2- 172.21.15.50:0/3909148501 >> v2:172.21.15.188:3300/0 conn(0x7f3f540d13d0 0x7f3f540b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.722+0000 7f3f58944640 1 -- 172.21.15.50:0/3909148501 >> v1:172.21.15.188:6789/0 conn(0x7f3f540d2e50 legacy=0x7f3f540d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:15.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.722+0000 7f3f58944640 1 -- 172.21.15.50:0/3909148501 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f540d3340 con 0x7f3f54161c80 2024-04-02T16:23:15.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.722+0000 7f3f59145640 1 --2- 172.21.15.50:0/3909148501 >> v2:172.21.15.188:3300/0 conn(0x7f3f540d13d0 0x7f3f540b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:15.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.722+0000 7f3f58944640 1 --2- 172.21.15.50:0/3909148501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540d2910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3f540b6ef0 tx=0x7f3f4c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de062754d3eb51cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:15.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f537fe640 1 -- 172.21.15.50:0/3909148501 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c02e070 con 0x7f3f54161c80 2024-04-02T16:23:15.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f537fe640 1 -- 172.21.15.50:0/3909148501 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4c037070 con 0x7f3f54161c80 2024-04-02T16:23:15.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f537fe640 1 -- 172.21.15.50:0/3909148501 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c031a00 con 0x7f3f54161c80 2024-04-02T16:23:15.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 -- 172.21.15.50:0/3909148501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 msgr2=0x7f3f540d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/3909148501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540d2910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3f540b6ef0 tx=0x7f3f4c02b880 comp rx=0 tx=0).stop 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 -- 172.21.15.50:0/3909148501 shutdown_connections 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/3909148501 >> v2:172.21.15.188:3300/0 conn(0x7f3f540d13d0 0x7f3f540b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/3909148501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540d2910 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 -- 172.21.15.50:0/3909148501 >> 172.21.15.50:0/3909148501 conn(0x7f3f5409d5e0 msgr2=0x7f3f540d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 -- 172.21.15.50:0/3909148501 shutdown_connections 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.723+0000 7f3f5aba8640 1 -- 172.21.15.50:0/3909148501 wait complete. 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.724+0000 7f3f5aba8640 1 Processor -- start 2024-04-02T16:23:15.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.724+0000 7f3f5aba8640 1 -- start start 2024-04-02T16:23:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f5aba8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f540d13d0 0x7f3f541558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f5aba8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f540d2e50 0x7f3f541f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f5aba8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f5aba8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f540b23a0 con 0x7f3f54161c80 2024-04-02T16:23:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f5aba8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f540d0390 con 0x7f3f540d2e50 2024-04-02T16:23:15.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f5aba8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3f540b2520 con 0x7f3f540d13d0 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f59145640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f59145640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40898/0 (socket says 172.21.15.50:40898) 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.725+0000 7f3f59145640 1 -- 172.21.15.50:0/2876597741 learned_addr learned my addr 172.21.15.50:0/2876597741 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f58944640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f540d13d0 0x7f3f541558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f53fff640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f540d2e50 0x7f3f541f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f59145640 1 -- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f540d13d0 msgr2=0x7f3f541558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f59145640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f540d13d0 0x7f3f541558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f59145640 1 -- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f540d2e50 msgr2=0x7f3f541f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f59145640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f540d2e50 0x7f3f541f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:15.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f59145640 1 -- 172.21.15.50:0/2876597741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f540b8110 con 0x7f3f54161c80 2024-04-02T16:23:15.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f59145640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540c87c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c7f0 tx=0x7f3f3c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:15.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f517fa640 1 -- 172.21.15.50:0/2876597741 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c015070 con 0x7f3f54161c80 2024-04-02T16:23:15.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f517fa640 1 -- 172.21.15.50:0/2876597741 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c004240 con 0x7f3f54161c80 2024-04-02T16:23:15.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f517fa640 1 -- 172.21.15.50:0/2876597741 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007620 con 0x7f3f54161c80 2024-04-02T16:23:15.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f540b83f0 con 0x7f3f54161c80 2024-04-02T16:23:15.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.726+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f540b89b0 con 0x7f3f54161c80 2024-04-02T16:23:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.727+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f54075760 con 0x7f3f54161c80 2024-04-02T16:23:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.728+0000 7f3f517fa640 1 -- 172.21.15.50:0/2876597741 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f3c0079d0 con 0x7f3f54161c80 2024-04-02T16:23:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.728+0000 7f3f517fa640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f3406c8f0 0x7f3f3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.728+0000 7f3f517fa640 1 -- 172.21.15.50:0/2876597741 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f3f3c08c060 con 0x7f3f54161c80 2024-04-02T16:23:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.728+0000 7f3f58944640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f3406c8f0 0x7f3f3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:15.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.729+0000 7f3f58944640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f3406c8f0 0x7f3f3406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c036000 tx=0x7f3f4c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:15.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.733+0000 7f3f517fa640 1 -- 172.21.15.50:0/2876597741 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3f3c059d00 con 0x7f3f54161c80 2024-04-02T16:23:15.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:15.956+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3f540d4a30 con 0x7f3f54161c80 2024-04-02T16:23:16.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.582+0000 7f3f517fa640 1 -- 172.21.15.50:0/2876597741 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f3f3c05e000 con 0x7f3f54161c80 2024-04-02T16:23:16.582 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 5 and data pool 6 2024-04-02T16:23:16.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f3406c8f0 msgr2=0x7f3f3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f3406c8f0 0x7f3f3406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c036000 tx=0x7f3f4c03d040 comp rx=0 tx=0).stop 2024-04-02T16:23:16.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 msgr2=0x7f3f540c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540c87c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c7f0 tx=0x7f3f3c00ccb0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 shutdown_connections 2024-04-02T16:23:16.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f3f3406c8f0 0x7f3f3406edb0 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c036000 tx=0x7f3f4c03d040 comp rx=0 tx=0).stop 2024-04-02T16:23:16.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f54161c80 0x7f3f540c87c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f540d2e50 0x7f3f541f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 --2- 172.21.15.50:0/2876597741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f540d13d0 0x7f3f541558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.586+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 >> 172.21.15.50:0/2876597741 conn(0x7f3f5409d5e0 msgr2=0x7f3f540b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:16.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.588+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 shutdown_connections 2024-04-02T16:23:16.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.592+0000 7f3f5aba8640 1 -- 172.21.15.50:0/2876597741 wait complete. 2024-04-02T16:23:16.615 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:23:16.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.721+0000 7f578bf9d640 1 Processor -- start 2024-04-02T16:23:16.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.721+0000 7f578bf9d640 1 -- start start 2024-04-02T16:23:16.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578bf9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:16.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578bf9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57840d0e90 0x7f57840d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:16.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578bf9d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57840ceda0 con 0x7f578415e510 2024-04-02T16:23:16.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578bf9d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f57840cf040 con 0x7f57840d0e90 2024-04-02T16:23:16.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578bf9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f57840cf320 con 0x7f57840d2f10 2024-04-02T16:23:16.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f5789538640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:16.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578a53a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57840d0e90 0x7f57840d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:16.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f5789d39640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f578415e510 0x7f57840d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.50:33552/0 (socket says 172.21.15.50:33552) 2024-04-02T16:23:16.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f5789d39640 1 -- 172.21.15.50:0/617830503 learned_addr learned my addr 172.21.15.50:0/617830503 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:16.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f5789538640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.50:59866/0 (socket says 172.21.15.50:59866) 2024-04-02T16:23:16.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578a53a640 1 -- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 msgr2=0x7f57840cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:16.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578a53a640 1 --2- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578a53a640 1 -- 172.21.15.50:0/617830503 >> v1:172.21.15.188:6789/0 conn(0x7f578415e510 legacy=0x7f57840d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f578a53a640 1 -- 172.21.15.50:0/617830503 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57840cf600 con 0x7f57840d0e90 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f5789538640 1 --2- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.722+0000 7f5788d37640 1 -- 172.21.15.50:0/617830503 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205860960 0 0) 0x7f57840ceda0 con 0x7f578415e510 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.723+0000 7f578a53a640 1 --2- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57840d0e90 0x7f57840d1320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f577c009d90 tx=0x7f577c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf6bd69a2426b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.723+0000 7f5788d37640 1 -- 172.21.15.50:0/617830503 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c039070 con 0x7f57840d0e90 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.723+0000 7f5788d37640 1 -- 172.21.15.50:0/617830503 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c02cbc0 con 0x7f57840d0e90 2024-04-02T16:23:16.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.723+0000 7f5788d37640 1 -- 172.21.15.50:0/617830503 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c034440 con 0x7f57840d0e90 2024-04-02T16:23:16.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 -- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57840d0e90 msgr2=0x7f57840d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 --2- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57840d0e90 0x7f57840d1320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f577c009d90 tx=0x7f577c02be10 comp rx=0 tx=0).stop 2024-04-02T16:23:16.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 -- 172.21.15.50:0/617830503 shutdown_connections 2024-04-02T16:23:16.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 --2- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57840d0e90 0x7f57840d1320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 --2- 172.21.15.50:0/617830503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 -- 172.21.15.50:0/617830503 >> 172.21.15.50:0/617830503 conn(0x7f578409d570 msgr2=0x7f57840d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:16.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 -- 172.21.15.50:0/617830503 shutdown_connections 2024-04-02T16:23:16.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.724+0000 7f578bf9d640 1 -- 172.21.15.50:0/617830503 wait complete. 2024-04-02T16:23:16.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.725+0000 7f578bf9d640 1 Processor -- start 2024-04-02T16:23:16.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.725+0000 7f578bf9d640 1 -- start start 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f578bf9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57840d0e90 0x7f57840b9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f578bf9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f578bf9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578415e510 0x7f57840b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f578bf9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57840d19e0 con 0x7f57840d0e90 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f578bf9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57840d1860 con 0x7f578415e510 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f578bf9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f57840b3ad0 con 0x7f57840d2f10 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57840d0e90 0x7f57840b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57840d0e90 0x7f57840b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40912/0 (socket says 172.21.15.50:40912) 2024-04-02T16:23:16.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 -- 172.21.15.50:0/1653613530 learned_addr learned my addr 172.21.15.50:0/1653613530 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789538640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f578a53a640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578415e510 0x7f57840b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 -- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 msgr2=0x7f57840b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 -- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578415e510 msgr2=0x7f57840b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578415e510 0x7f57840b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 -- 172.21.15.50:0/1653613530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57840b7a10 con 0x7f57840d0e90 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.726+0000 7f5789d39640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57840d0e90 0x7f57840b9640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f578000c430 tx=0x7f578000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.727+0000 7f577a7fc640 1 -- 172.21.15.50:0/1653613530 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57800074e0 con 0x7f57840d0e90 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.727+0000 7f577a7fc640 1 -- 172.21.15.50:0/1653613530 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57800047e0 con 0x7f57840d0e90 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.727+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57840b8e00 con 0x7f57840d0e90 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.727+0000 7f577a7fc640 1 -- 172.21.15.50:0/1653613530 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780004bd0 con 0x7f57840d0e90 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.727+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57840b93c0 con 0x7f57840d0e90 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.728+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57840756f0 con 0x7f57840d0e90 2024-04-02T16:23:16.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.728+0000 7f577a7fc640 1 -- 172.21.15.50:0/1653613530 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5780011070 con 0x7f57840d0e90 2024-04-02T16:23:16.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.728+0000 7f577a7fc640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f576806c8f0 0x7f576806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:16.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.729+0000 7f577a7fc640 1 -- 172.21.15.50:0/1653613530 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f578008b8a0 con 0x7f57840d0e90 2024-04-02T16:23:16.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.729+0000 7f5789538640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f576806c8f0 0x7f576806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:16.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.729+0000 7f5789538640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f576806c8f0 0x7f576806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5774009560 tx=0x7f5774009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:16.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.733+0000 7f577a7fc640 1 -- 172.21.15.50:0/1653613530 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5780059470 con 0x7f57840d0e90 2024-04-02T16:23:16.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.918+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57840ce6c0 con 0x7f57840d0e90 2024-04-02T16:23:16.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.919+0000 7f577a7fc640 1 -- 172.21.15.50:0/1653613530 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16943 (secure 0 0 0) 0x7f578005d770 con 0x7f57840d0e90 2024-04-02T16:23:16.919 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:16.919 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":38,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:23:15.957618+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:23:13.826831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-02T16:23:14.911147+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f576806c8f0 msgr2=0x7f576806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f576806c8f0 0x7f576806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5774009560 tx=0x7f5774009190 comp rx=0 tx=0).stop 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57840d0e90 msgr2=0x7f57840b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57840d0e90 0x7f57840b9640 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f578000c430 tx=0x7f578000c8f0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 shutdown_connections 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f578415e510 0x7f57840b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57840d2f10 0x7f57840b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f576806c8f0 0x7f576806edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 --2- 172.21.15.50:0/1653613530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57840d0e90 0x7f57840b9640 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:16.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 >> 172.21.15.50:0/1653613530 conn(0x7f578409d570 msgr2=0x7f57840b51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:16.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 shutdown_connections 2024-04-02T16:23:16.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:16.921+0000 7f578bf9d640 1 -- 172.21.15.50:0/1653613530 wait complete. 2024-04-02T16:23:16.932 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:23:17.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.033+0000 7fc828e3a640 1 Processor -- start 2024-04-02T16:23:17.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.033+0000 7fc828e3a640 1 -- start start 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc828e3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc8240b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc828e3a640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc8240b3b40 0x7fc8240b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc828e3a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8240b4a50 0x7fc824142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc828e3a640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fc8240b4ee0 con 0x7fc8240b3b40 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc828e3a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc8240b5400 con 0x7fc8240b4a50 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc828e3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc8240b24e0 con 0x7fc8240b1bc0 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc821d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc8240b3b40 0x7fc8240b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc821d9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fc8240b3b40 0x7fc8240b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40920/0 (socket says 172.21.15.50:40920) 2024-04-02T16:23:17.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.034+0000 7fc821d9b640 1 -- 172.21.15.50:0/2264140922 learned_addr learned my addr 172.21.15.50:0/2264140922 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc82259c640 1 --2- 172.21.15.50:0/2264140922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc8240b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc821d9b640 1 -- 172.21.15.50:0/2264140922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 msgr2=0x7fc8240b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc821d9b640 1 --2- 172.21.15.50:0/2264140922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc8240b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc822d9d640 1 --2- 172.21.15.50:0/2264140922 >> v2:172.21.15.188:3300/0 conn(0x7fc8240b4a50 0x7fc824142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc821d9b640 1 -- 172.21.15.50:0/2264140922 >> v2:172.21.15.188:3300/0 conn(0x7fc8240b4a50 msgr2=0x7fc824142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc821d9b640 1 --2- 172.21.15.50:0/2264140922 >> v2:172.21.15.188:3300/0 conn(0x7fc8240b4a50 0x7fc824142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc821d9b640 1 -- 172.21.15.50:0/2264140922 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8240b5120 con 0x7fc8240b3b40 2024-04-02T16:23:17.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc822d9d640 1 --2- 172.21.15.50:0/2264140922 >> v2:172.21.15.188:3300/0 conn(0x7fc8240b4a50 0x7fc824142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:17.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.035+0000 7fc821d9b640 1 --2- 172.21.15.50:0/2264140922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b3b40 0x7fc8240b4510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc824160e30 tx=0x7fc80c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54447a85e67f59c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc82159a640 1 -- 172.21.15.50:0/2264140922 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0317d0 con 0x7fc8240b3b40 2024-04-02T16:23:17.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc82159a640 1 -- 172.21.15.50:0/2264140922 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80c037070 con 0x7fc8240b3b40 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc82159a640 1 -- 172.21.15.50:0/2264140922 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c031b90 con 0x7fc8240b3b40 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 -- 172.21.15.50:0/2264140922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b3b40 msgr2=0x7fc8240b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 --2- 172.21.15.50:0/2264140922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b3b40 0x7fc8240b4510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc824160e30 tx=0x7fc80c02b870 comp rx=0 tx=0).stop 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 -- 172.21.15.50:0/2264140922 shutdown_connections 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 --2- 172.21.15.50:0/2264140922 >> v2:172.21.15.188:3300/0 conn(0x7fc8240b4a50 0x7fc824142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 --2- 172.21.15.50:0/2264140922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b3b40 0x7fc8240b4510 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 --2- 172.21.15.50:0/2264140922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc8240b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 -- 172.21.15.50:0/2264140922 >> 172.21.15.50:0/2264140922 conn(0x7fc82406ebf0 msgr2=0x7fc82406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 -- 172.21.15.50:0/2264140922 shutdown_connections 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.036+0000 7fc828e3a640 1 -- 172.21.15.50:0/2264140922 wait complete. 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.037+0000 7fc828e3a640 1 Processor -- start 2024-04-02T16:23:17.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.037+0000 7fc828e3a640 1 -- start start 2024-04-02T16:23:17.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc828e3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc824136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc828e3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8240b3b40 0x7fc8241454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc828e3a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b4a50 0x7fc824145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc828e3a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240b3620 con 0x7fc8240b4a50 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc828e3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240b3340 con 0x7fc8240b3b40 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc828e3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc824142b00 con 0x7fc8240b1bc0 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc822d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b4a50 0x7fc824145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc822d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b4a50 0x7fc824145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40934/0 (socket says 172.21.15.50:40934) 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.038+0000 7fc822d9d640 1 -- 172.21.15.50:0/3427284324 learned_addr learned my addr 172.21.15.50:0/3427284324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc82259c640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc824136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc821d9b640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8240b3b40 0x7fc8241454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc822d9d640 1 -- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 msgr2=0x7fc824136640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc822d9d640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc824136640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc822d9d640 1 -- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8240b3b40 msgr2=0x7fc8241454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc822d9d640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8240b3b40 0x7fc8241454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc822d9d640 1 -- 172.21.15.50:0/3427284324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc824136b80 con 0x7fc8240b4a50 2024-04-02T16:23:17.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc822d9d640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b4a50 0x7fc824145a30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cd80 tx=0x7fc81400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc81affd640 1 -- 172.21.15.50:0/3427284324 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400f680 con 0x7fc8240b4a50 2024-04-02T16:23:17.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc81affd640 1 -- 172.21.15.50:0/3427284324 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc81400f840 con 0x7fc8240b4a50 2024-04-02T16:23:17.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc824156de0 con 0x7fc8240b4a50 2024-04-02T16:23:17.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc81affd640 1 -- 172.21.15.50:0/3427284324 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400fc30 con 0x7fc8240b4a50 2024-04-02T16:23:17.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.039+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8241573a0 con 0x7fc8240b4a50 2024-04-02T16:23:17.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.040+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8240756f0 con 0x7fc8240b4a50 2024-04-02T16:23:17.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.041+0000 7fc81affd640 1 -- 172.21.15.50:0/3427284324 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8140213f0 con 0x7fc8240b4a50 2024-04-02T16:23:17.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.041+0000 7fc81affd640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc80406c8f0 0x7fc80406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.041+0000 7fc81affd640 1 -- 172.21.15.50:0/3427284324 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc81408ca00 con 0x7fc8240b4a50 2024-04-02T16:23:17.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.041+0000 7fc82259c640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc80406c8f0 0x7fc80406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.042+0000 7fc82259c640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc80406c8f0 0x7fc80406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00a990 tx=0x7fc81c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.045+0000 7fc81affd640 1 -- 172.21.15.50:0/3427284324 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc814014030 con 0x7fc8240b4a50 2024-04-02T16:23:17.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.232+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8240cef50 con 0x7fc8240b4a50 2024-04-02T16:23:17.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.233+0000 7fc81affd640 1 -- 172.21.15.50:0/3427284324 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16943 (secure 0 0 0) 0x7fc81405a6e0 con 0x7fc8240b4a50 2024-04-02T16:23:17.234 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:17.234 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":38,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:23:15.957618+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:23:13.826831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-02T16:23:14.911147+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc80406c8f0 msgr2=0x7fc80406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc80406c8f0 0x7fc80406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00a990 tx=0x7fc81c008040 comp rx=0 tx=0).stop 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b4a50 msgr2=0x7fc824145a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b4a50 0x7fc824145a30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cd80 tx=0x7fc81400a650 comp rx=0 tx=0).stop 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 shutdown_connections 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fc80406c8f0 0x7fc80406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8240b4a50 0x7fc824145a30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8240b3b40 0x7fc8241454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 --2- 172.21.15.50:0/3427284324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8240b1bc0 0x7fc824136640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 >> 172.21.15.50:0/3427284324 conn(0x7fc82406ebf0 msgr2=0x7fc824160710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:17.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 shutdown_connections 2024-04-02T16:23:17.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.235+0000 7fc828e3a640 1 -- 172.21.15.50:0/3427284324 wait complete. 2024-04-02T16:23:17.247 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T16:23:17.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.354+0000 7fd6f5847640 1 Processor -- start 2024-04-02T16:23:17.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.354+0000 7fd6f5847640 1 -- start start 2024-04-02T16:23:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6f5847640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd6f015e710 0x7fd6f00b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6f5847640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f009aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6f5847640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6f009b690 con 0x7fd6f00b4840 2024-04-02T16:23:17.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6f5847640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f006c840 con 0x7fd6f015e710 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6f5847640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f006ca90 con 0x7fd6f009aa40 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6eeffd640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fd6f015e710 0x7fd6f00b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6e7fff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fd6f00b4840 0x7fd6f00d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42436/0 (socket says 172.21.15.50:42436) 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6e7fff640 1 -- 172.21.15.50:0/4059259412 learned_addr learned my addr 172.21.15.50:0/4059259412 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6ef7fe640 1 --2- 172.21.15.50:0/4059259412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f009aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6eeffd640 1 -- 172.21.15.50:0/4059259412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 msgr2=0x7fd6f009aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6eeffd640 1 --2- 172.21.15.50:0/4059259412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f009aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6eeffd640 1 -- 172.21.15.50:0/4059259412 >> v1:172.21.15.50:6789/0 conn(0x7fd6f00b4840 legacy=0x7fd6f00d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6eeffd640 1 -- 172.21.15.50:0/4059259412 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f006d0e0 con 0x7fd6f015e710 2024-04-02T16:23:17.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.355+0000 7fd6ee7fc640 1 -- 172.21.15.50:0/4059259412 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1429247719 0 0) 0x7fd6f009b690 con 0x7fd6f00b4840 2024-04-02T16:23:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.356+0000 7fd6eeffd640 1 --2- 172.21.15.50:0/4059259412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f015e710 0x7fd6f00b3220 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00097a0 tx=0x7fd6e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784396645980aa28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.356+0000 7fd6ee7fc640 1 -- 172.21.15.50:0/4059259412 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00315a0 con 0x7fd6f015e710 2024-04-02T16:23:17.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.356+0000 7fd6ee7fc640 1 -- 172.21.15.50:0/4059259412 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e0037070 con 0x7fd6f015e710 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.356+0000 7fd6ee7fc640 1 -- 172.21.15.50:0/4059259412 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0031960 con 0x7fd6f015e710 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 -- 172.21.15.50:0/4059259412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f015e710 msgr2=0x7fd6f00b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 --2- 172.21.15.50:0/4059259412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f015e710 0x7fd6f00b3220 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00097a0 tx=0x7fd6e002b840 comp rx=0 tx=0).stop 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 -- 172.21.15.50:0/4059259412 shutdown_connections 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 --2- 172.21.15.50:0/4059259412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f009aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 --2- 172.21.15.50:0/4059259412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f015e710 0x7fd6f00b3220 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 -- 172.21.15.50:0/4059259412 >> 172.21.15.50:0/4059259412 conn(0x7fd6f00b3b10 msgr2=0x7fd6f00b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 -- 172.21.15.50:0/4059259412 shutdown_connections 2024-04-02T16:23:17.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.357+0000 7fd6f5847640 1 -- 172.21.15.50:0/4059259412 wait complete. 2024-04-02T16:23:17.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.358+0000 7fd6f5847640 1 Processor -- start 2024-04-02T16:23:17.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.358+0000 7fd6f5847640 1 -- start start 2024-04-02T16:23:17.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f00c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f00b4840 0x7fd6f0159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6f015e710 0x7fd6f015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f006d6a0 con 0x7fd6f00b4840 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f006cd70 con 0x7fd6f015e710 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f006d820 con 0x7fd6f009aa40 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f00b4840 0x7fd6f0159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f00b4840 0x7fd6f0159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40938/0 (socket says 172.21.15.50:40938) 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 -- 172.21.15.50:0/743696436 learned_addr learned my addr 172.21.15.50:0/743696436 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6eeffd640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f00c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 -- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 msgr2=0x7fd6f00c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f00c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 -- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6f015e710 msgr2=0x7fd6f015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6ef7fe640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6f015e710 0x7fd6f015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6f015e710 0x7fd6f015f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 -- 172.21.15.50:0/743696436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f00c61a0 con 0x7fd6f00b4840 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6eeffd640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f00c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6e7fff640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f00b4840 0x7fd6f0159650 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800c4d0 tx=0x7fd6d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6ecff9640 1 -- 172.21.15.50:0/743696436 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8018070 con 0x7fd6f00b4840 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6ecff9640 1 -- 172.21.15.50:0/743696436 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d8007500 con 0x7fd6f00b4840 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f00c6480 con 0x7fd6f00b4840 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6ecff9640 1 -- 172.21.15.50:0/743696436 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8004a20 con 0x7fd6f00b4840 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.359+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f00c6a40 con 0x7fd6f00b4840 2024-04-02T16:23:17.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.360+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f00756f0 con 0x7fd6f00b4840 2024-04-02T16:23:17.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.361+0000 7fd6ecff9640 1 -- 172.21.15.50:0/743696436 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6d801b070 con 0x7fd6f00b4840 2024-04-02T16:23:17.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.361+0000 7fd6ecff9640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd6d006c8f0 0x7fd6d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.361+0000 7fd6ecff9640 1 -- 172.21.15.50:0/743696436 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd6d808aff0 con 0x7fd6f00b4840 2024-04-02T16:23:17.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.361+0000 7fd6eeffd640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd6d006c8f0 0x7fd6d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.362+0000 7fd6eeffd640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd6d006c8f0 0x7fd6d006edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0036000 tx=0x7fd6e0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.366+0000 7fd6ecff9640 1 -- 172.21.15.50:0/743696436 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd6d80592c0 con 0x7fd6f00b4840 2024-04-02T16:23:17.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.590+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd6f00b7fa0 con 0x7fd6f00b4840 2024-04-02T16:23:17.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.603+0000 7fd6ecff9640 1 -- 172.21.15.50:0/743696436 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fd6d8058bc0 con 0x7fd6f00b4840 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.605+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd6d006c8f0 msgr2=0x7fd6d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.605+0000 7fd6f5847640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd6d006c8f0 0x7fd6d006edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0036000 tx=0x7fd6e0008030 comp rx=0 tx=0).stop 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.605+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f00b4840 msgr2=0x7fd6f0159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.605+0000 7fd6f5847640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f00b4840 0x7fd6f0159650 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800c4d0 tx=0x7fd6d800c990 comp rx=0 tx=0).stop 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 shutdown_connections 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6f015e710 0x7fd6f015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fd6d006c8f0 0x7fd6d006edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6f00b4840 0x7fd6f0159650 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 --2- 172.21.15.50:0/743696436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6f009aa40 0x7fd6f00c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 >> 172.21.15.50:0/743696436 conn(0x7fd6f00b3b10 msgr2=0x7fd6f00b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:17.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 shutdown_connections 2024-04-02T16:23:17.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.606+0000 7fd6f5847640 1 -- 172.21.15.50:0/743696436 wait complete. 2024-04-02T16:23:17.616 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-02T16:23:17.616 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-02T16:23:17.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.721+0000 7f1b58db5640 1 Processor -- start 2024-04-02T16:23:17.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.721+0000 7f1b58db5640 1 -- start start 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b58db5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 0x7f1b540d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b58db5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b540d4c90 0x7f1b540b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b58db5640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5415e9f0 con 0x7f1b5415e610 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b58db5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1b540d5120 con 0x7f1b540d4c90 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b58db5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b540d3210 con 0x7f1b540d1350 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b5259c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f1b5415e610 0x7f1b540d0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42450/0 (socket says 172.21.15.50:42450) 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b5259c640 1 -- 172.21.15.50:0/1090479652 learned_addr learned my addr 172.21.15.50:0/1090479652 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b4bfff640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 0x7f1b540d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/1090479652 >> v2:172.21.15.188:3300/0 conn(0x7f1b540d4c90 0x7f1b540b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380187770 0 0) 0x7f1b5415e9f0 con 0x7f1b5415e610 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b40003520 con 0x7f1b5415e610 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2522880918 0 0) 0x7f1b40003520 con 0x7f1b5415e610 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 msgr2=0x7f1b540d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 0x7f1b540d2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 >> v2:172.21.15.188:3300/0 conn(0x7f1b540d4c90 msgr2=0x7f1b540b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 --2- 172.21.15.50:0/1090479652 >> v2:172.21.15.188:3300/0 conn(0x7f1b540d4c90 0x7f1b540b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b540d2f30 con 0x7f1b5415e610 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f1b44001670 con 0x7f1b5415e610 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 >> v1:172.21.15.50:6789/0 conn(0x7f1b5415e610 legacy=0x7f1b540d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 0x7f1b400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b40003f30 0x7f1b40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b540d5120 con 0x7f1b40004d40 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b540d3210 con 0x7f1b40003f30 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.722+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4000b9c0 con 0x7f1b540d1350 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b5259c640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 0x7f1b400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b4bfff640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b40003f30 0x7f1b40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b52d9d640 1 -- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 msgr2=0x7f1b400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540d1350 0x7f1b400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b52d9d640 1 -- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b40003f30 msgr2=0x7f1b40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b40003f30 0x7f1b40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b52d9d640 1 -- 172.21.15.50:0/1090479652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b40003520 con 0x7f1b40004d40 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.723+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009fc0 tx=0x7f1b40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c6af850f418865 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c039070 con 0x7f1b40004d40 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c02cc40 con 0x7f1b40004d40 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b51d9b640 1 -- 172.21.15.50:0/1090479652 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c009c10 con 0x7f1b40004d40 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b58db5640 1 -- 172.21.15.50:0/1090479652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b40004d40 msgr2=0x7f1b400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b58db5640 1 --2- 172.21.15.50:0/1090479652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b40004d40 0x7f1b400051c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009fc0 tx=0x7f1b40005830 comp rx=0 tx=0).stop 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b52d9d640 1 -- 172.21.15.50:0/1090479652 reap_dead start 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b58db5640 1 -- 172.21.15.50:0/1090479652 shutdown_connections 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b58db5640 1 -- 172.21.15.50:0/1090479652 >> 172.21.15.50:0/1090479652 conn(0x7f1b5409d550 msgr2=0x7f1b540d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b58db5640 1 -- 172.21.15.50:0/1090479652 shutdown_connections 2024-04-02T16:23:17.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.724+0000 7f1b58db5640 1 -- 172.21.15.50:0/1090479652 wait complete. 2024-04-02T16:23:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 Processor -- start 2024-04-02T16:23:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 -- start start 2024-04-02T16:23:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b54153be0 0x7f1b540cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540cdeb0 0x7f1b54160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b54161140 0x7f1b541f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b540b4310 con 0x7f1b54161140 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b540b4030 con 0x7f1b54153be0 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b58db5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b540d2850 con 0x7f1b540cdeb0 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b52d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b54161140 0x7f1b541f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b52d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b54161140 0x7f1b541f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40950/0 (socket says 172.21.15.50:40950) 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.726+0000 7f1b52d9d640 1 -- 172.21.15.50:0/2139193955 learned_addr learned my addr 172.21.15.50:0/2139193955 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b4bfff640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540cdeb0 0x7f1b54160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b5259c640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b54153be0 0x7f1b540cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b52d9d640 1 -- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540cdeb0 msgr2=0x7f1b54160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540cdeb0 0x7f1b54160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b52d9d640 1 -- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b54153be0 msgr2=0x7f1b540cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b54153be0 0x7f1b540cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b52d9d640 1 -- 172.21.15.50:0/2139193955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b540b94b0 con 0x7f1b54161140 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b52d9d640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b54161140 0x7f1b541f76e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00e230 tx=0x7f1b4c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b4affd640 1 -- 172.21.15.50:0/2139193955 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c039030 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b4affd640 1 -- 172.21.15.50:0/2139193955 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c03a040 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b54160440 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b4affd640 1 -- 172.21.15.50:0/2139193955 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c009c10 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.727+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b540b8ef0 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.728+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b540756f0 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.729+0000 7f1b4affd640 1 -- 172.21.15.50:0/2139193955 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b4c005d10 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.729+0000 7f1b4affd640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b3406c920 0x7f1b3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.729+0000 7f1b4affd640 1 -- 172.21.15.50:0/2139193955 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1b4c0ad640 con 0x7f1b54161140 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.729+0000 7f1b5259c640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b3406c920 0x7f1b3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:17.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.730+0000 7f1b5259c640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b3406c920 0x7f1b3406ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b44004840 tx=0x7f1b44009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:17.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.733+0000 7f1b4affd640 1 -- 172.21.15.50:0/2139193955 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1b4c07b970 con 0x7f1b54161140 2024-04-02T16:23:17.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:17.957+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1b5415cce0 con 0x7f1b54161140 2024-04-02T16:23:18.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.608+0000 7f1b4affd640 1 -- 172.21.15.50:0/2139193955 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f1b4c07b270 con 0x7f1b54161140 2024-04-02T16:23:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b3406c920 msgr2=0x7f1b3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b3406c920 0x7f1b3406ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b44004840 tx=0x7f1b44009040 comp rx=0 tx=0).stop 2024-04-02T16:23:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b54161140 msgr2=0x7f1b541f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:18.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b54161140 0x7f1b541f76e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00e230 tx=0x7f1b4c009750 comp rx=0 tx=0).stop 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 shutdown_connections 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b3406c920 0x7f1b3406ede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b54161140 0x7f1b541f76e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b540cdeb0 0x7f1b54160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 --2- 172.21.15.50:0/2139193955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b54153be0 0x7f1b540cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 >> 172.21.15.50:0/2139193955 conn(0x7f1b5409d550 msgr2=0x7f1b540b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 shutdown_connections 2024-04-02T16:23:18.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.610+0000 7f1b58db5640 1 -- 172.21.15.50:0/2139193955 wait complete. 2024-04-02T16:23:18.624 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-02T16:23:18.624 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:23:18.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.731+0000 7f9ad958d640 1 Processor -- start 2024-04-02T16:23:18.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.731+0000 7f9ad958d640 1 -- start start 2024-04-02T16:23:18.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad958d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ad41584a0 0x7f9ad415b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad958d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad415e2b0 0x7f9ad415e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad958d640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9ad40b1bc0 0x7f9ad40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad958d640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad415acc0 con 0x7f9ad40b1bc0 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad958d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad415bc50 con 0x7f9ad41584a0 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad958d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad415ed10 con 0x7f9ad415e2b0 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9ad40b1bc0 0x7f9ad40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad37fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9ad40b1bc0 0x7f9ad40b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40954/0 (socket says 172.21.15.50:40954) 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9ad37fe640 1 -- 172.21.15.50:0/2913192196 learned_addr learned my addr 172.21.15.50:0/2913192196 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.732+0000 7f9acbfff640 1 --2- 172.21.15.50:0/2913192196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad415e2b0 0x7f9ad415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad2ffd640 1 --2- 172.21.15.50:0/2913192196 >> v2:172.21.15.188:3300/0 conn(0x7f9ad41584a0 0x7f9ad415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad37fe640 1 -- 172.21.15.50:0/2913192196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad415e2b0 msgr2=0x7f9ad415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad37fe640 1 --2- 172.21.15.50:0/2913192196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad415e2b0 0x7f9ad415e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad37fe640 1 -- 172.21.15.50:0/2913192196 >> v2:172.21.15.188:3300/0 conn(0x7f9ad41584a0 msgr2=0x7f9ad415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad37fe640 1 --2- 172.21.15.50:0/2913192196 >> v2:172.21.15.188:3300/0 conn(0x7f9ad41584a0 0x7f9ad415b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad37fe640 1 -- 172.21.15.50:0/2913192196 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad415eff0 con 0x7f9ad40b1bc0 2024-04-02T16:23:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad2ffd640 1 --2- 172.21.15.50:0/2913192196 >> v2:172.21.15.188:3300/0 conn(0x7f9ad41584a0 0x7f9ad415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad37fe640 1 --2- 172.21.15.50:0/2913192196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad40b1bc0 0x7f9ad40b2070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9ad406f660 tx=0x7f9acc02cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22d52cd927bcb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.733+0000 7f9ad27fc640 1 -- 172.21.15.50:0/2913192196 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc02d8f0 con 0x7f9ad40b1bc0 2024-04-02T16:23:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad27fc640 1 -- 172.21.15.50:0/2913192196 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9acc02da90 con 0x7f9ad40b1bc0 2024-04-02T16:23:18.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad27fc640 1 -- 172.21.15.50:0/2913192196 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc02de60 con 0x7f9ad40b1bc0 2024-04-02T16:23:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 -- 172.21.15.50:0/2913192196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad40b1bc0 msgr2=0x7f9ad40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 --2- 172.21.15.50:0/2913192196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad40b1bc0 0x7f9ad40b2070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9ad406f660 tx=0x7f9acc02cf50 comp rx=0 tx=0).stop 2024-04-02T16:23:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 -- 172.21.15.50:0/2913192196 shutdown_connections 2024-04-02T16:23:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 --2- 172.21.15.50:0/2913192196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad40b1bc0 0x7f9ad40b2070 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 --2- 172.21.15.50:0/2913192196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad415e2b0 0x7f9ad415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 --2- 172.21.15.50:0/2913192196 >> v2:172.21.15.188:3300/0 conn(0x7f9ad41584a0 0x7f9ad415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 -- 172.21.15.50:0/2913192196 >> 172.21.15.50:0/2913192196 conn(0x7f9ad406ebf0 msgr2=0x7f9ad406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 -- 172.21.15.50:0/2913192196 shutdown_connections 2024-04-02T16:23:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.734+0000 7f9ad958d640 1 -- 172.21.15.50:0/2913192196 wait complete. 2024-04-02T16:23:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.735+0000 7f9ad958d640 1 Processor -- start 2024-04-02T16:23:18.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.735+0000 7f9ad958d640 1 -- start start 2024-04-02T16:23:18.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad958d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad40b1bc0 0x7f9ad4153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:18.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad958d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad41584a0 0x7f9ad414bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:18.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad958d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad415e2b0 0x7f9ad40b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:18.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad958d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad40b4a80 con 0x7f9ad415e2b0 2024-04-02T16:23:18.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad958d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad415b1f0 con 0x7f9ad41584a0 2024-04-02T16:23:18.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad958d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad4154df0 con 0x7f9ad40b1bc0 2024-04-02T16:23:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad415e2b0 0x7f9ad40b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad415e2b0 0x7f9ad40b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40968/0 (socket says 172.21.15.50:40968) 2024-04-02T16:23:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 -- 172.21.15.50:0/1733819301 learned_addr learned my addr 172.21.15.50:0/1733819301 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 -- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad40b1bc0 msgr2=0x7f9ad4153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad2ffd640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad40b1bc0 0x7f9ad4153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9acbfff640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad41584a0 0x7f9ad414bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:18.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad40b1bc0 0x7f9ad4153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 -- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad41584a0 msgr2=0x7f9ad414bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad41584a0 0x7f9ad414bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.736+0000 7f9ad37fe640 1 -- 172.21.15.50:0/1733819301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad41537d0 con 0x7f9ad415e2b0 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.737+0000 7f9ad37fe640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad415e2b0 0x7f9ad40b2e50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9acc037040 tx=0x7f9acc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.737+0000 7f9acb7fe640 1 -- 172.21.15.50:0/1733819301 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc044020 con 0x7f9ad415e2b0 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.737+0000 7f9acb7fe640 1 -- 172.21.15.50:0/1733819301 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9acc040670 con 0x7f9ad415e2b0 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.737+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad4154830 con 0x7f9ad415e2b0 2024-04-02T16:23:18.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.737+0000 7f9acb7fe640 1 -- 172.21.15.50:0/1733819301 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00a620 con 0x7f9ad415e2b0 2024-04-02T16:23:18.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.737+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad4150820 con 0x7f9ad415e2b0 2024-04-02T16:23:18.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.738+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad40756f0 con 0x7f9ad415e2b0 2024-04-02T16:23:18.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.738+0000 7f9acb7fe640 1 -- 172.21.15.50:0/1733819301 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9acc047020 con 0x7f9ad415e2b0 2024-04-02T16:23:18.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.739+0000 7f9acb7fe640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9ab406c8f0 0x7f9ab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:18.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.739+0000 7f9acb7fe640 1 -- 172.21.15.50:0/1733819301 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9acc0ae950 con 0x7f9ad415e2b0 2024-04-02T16:23:18.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.739+0000 7f9ad2ffd640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9ab406c8f0 0x7f9ab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:18.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.740+0000 7f9ad2ffd640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9ab406c8f0 0x7f9ab406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b840 tx=0x7f9ac4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:18.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.743+0000 7f9acb7fe640 1 -- 172.21.15.50:0/1733819301 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9acc04e030 con 0x7f9ad415e2b0 2024-04-02T16:23:18.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.965+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ad4159370 con 0x7f9ad415e2b0 2024-04-02T16:23:18.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.966+0000 7f9acb7fe640 1 -- 172.21.15.50:0/1733819301 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f9acc07cbf0 con 0x7f9ad415e2b0 2024-04-02T16:23:18.966 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:18.966 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/4131511681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":4131511681},{"type":"v1","addr":"172.21.15.188:6837","nonce":4131511681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1831814256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1831814256},{"type":"v1","addr":"172.21.15.188:6836","nonce":1831814256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/1153018762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1153018762},{"type":"v1","addr":"172.21.15.50:6834","nonce":1153018762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:23:15.957913+0000","modified":"2024-04-02T16:23:17.958571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/3098609271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":3098609271},{"type":"v1","addr":"172.21.15.50:6835","nonce":3098609271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-04-02T16:23:18.967 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 16 2024-04-02T16:23:18.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9ab406c8f0 msgr2=0x7f9ab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:18.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9ab406c8f0 0x7f9ab406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b840 tx=0x7f9ac4008040 comp rx=0 tx=0).stop 2024-04-02T16:23:18.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad415e2b0 msgr2=0x7f9ad40b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:18.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad415e2b0 0x7f9ad40b2e50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9acc037040 tx=0x7f9acc004a40 comp rx=0 tx=0).stop 2024-04-02T16:23:18.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 shutdown_connections 2024-04-02T16:23:18.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9ab406c8f0 0x7f9ab406edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ad415e2b0 0x7f9ad40b2e50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad41584a0 0x7f9ad414bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 --2- 172.21.15.50:0/1733819301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ad40b1bc0 0x7f9ad4153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:18.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 >> 172.21.15.50:0/1733819301 conn(0x7f9ad406ebf0 msgr2=0x7f9ad4072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:18.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 shutdown_connections 2024-04-02T16:23:18.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:18.968+0000 7f9ad958d640 1 -- 172.21.15.50:0/1733819301 wait complete. 2024-04-02T16:23:18.979 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:23:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.080+0000 7f6027cf2640 1 Processor -- start 2024-04-02T16:23:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.080+0000 7f6027cf2640 1 -- start start 2024-04-02T16:23:19.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6027cf2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f60200d2f10 0x7f60200cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6027cf2640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60200ceda0 con 0x7f602015e510 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6027cf2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60200cf040 con 0x7f60200d0e90 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6027cf2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f60200d19c0 con 0x7f60200d2f10 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6025a8e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f602015e510 0x7f60200d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42454/0 (socket says 172.21.15.50:42454) 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6025a8e640 1 -- 172.21.15.50:0/40776718 learned_addr learned my addr 172.21.15.50:0/40776718 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f602528d640 1 --2- 172.21.15.50:0/40776718 >> v2:172.21.15.188:3300/0 conn(0x7f60200d2f10 0x7f60200cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203328368 0 0) 0x7f60200ceda0 con 0x7f602015e510 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f600c003520 con 0x7f602015e510 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2196120443 0 0) 0x7f600c003520 con 0x7f602015e510 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 >> v2:172.21.15.188:3300/0 conn(0x7f60200d2f10 msgr2=0x7f60200cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 --2- 172.21.15.50:0/40776718 >> v2:172.21.15.188:3300/0 conn(0x7f60200d2f10 0x7f60200cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 >> v1:172.21.15.188:6789/0 conn(0x7f60200d0e90 legacy=0x7f60200d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60200cf320 con 0x7f602015e510 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f601c001690 con 0x7f602015e510 2024-04-02T16:23:19.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 >> v1:172.21.15.50:6789/0 conn(0x7f602015e510 legacy=0x7f60200d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c0039b0 0x7f600c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c004800 0x7f600c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.081+0000 7f6024a8c640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f600c0051c0 0x7f600c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60200d19c0 con 0x7f600c0051c0 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f600c00c160 con 0x7f600c0039b0 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f600c00c360 con 0x7f600c004800 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602628f640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f600c0051c0 0x7f600c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f6025a8e640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c0039b0 0x7f600c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602528d640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c004800 0x7f600c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602628f640 1 -- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c004800 msgr2=0x7f600c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602628f640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c004800 0x7f600c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602628f640 1 -- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c0039b0 msgr2=0x7f600c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602628f640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c0039b0 0x7f600c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602628f640 1 -- 172.21.15.50:0/40776718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c003520 con 0x7f600c0051c0 2024-04-02T16:23:19.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602528d640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c004800 0x7f600c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.082+0000 7f602628f640 1 -- 172.21.15.50:0/40776718 reap_dead start 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f602628f640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f600c0051c0 0x7f600c00b9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6018001e50 tx=0x7f600c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fa856d4eb40cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018001a70 con 0x7f600c0051c0 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6018003bb0 con 0x7f600c0051c0 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6024a8c640 1 -- 172.21.15.50:0/40776718 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180050d0 con 0x7f600c0051c0 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6027cf2640 1 -- 172.21.15.50:0/40776718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f600c0051c0 msgr2=0x7f600c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6027cf2640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f600c0051c0 0x7f600c00b9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6018001e50 tx=0x7f600c005830 comp rx=0 tx=0).stop 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6027cf2640 1 -- 172.21.15.50:0/40776718 shutdown_connections 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6027cf2640 1 --2- 172.21.15.50:0/40776718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f600c0051c0 0x7f600c00b9c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6027cf2640 1 -- 172.21.15.50:0/40776718 >> 172.21.15.50:0/40776718 conn(0x7f602009d570 msgr2=0x7f60200d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6027cf2640 1 -- 172.21.15.50:0/40776718 shutdown_connections 2024-04-02T16:23:19.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.083+0000 7f6027cf2640 1 -- 172.21.15.50:0/40776718 wait complete. 2024-04-02T16:23:19.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.084+0000 7f6027cf2640 1 Processor -- start 2024-04-02T16:23:19.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6027cf2640 1 -- start start 2024-04-02T16:23:19.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6027cf2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0051c0 0x7f602015c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6027cf2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f602009dfe0 0x7f602009e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6027cf2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60200a0040 0x7f60200a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6027cf2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60200cfe00 con 0x7f60200a0040 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6027cf2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f602015e8f0 con 0x7f602009dfe0 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6027cf2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60200cff80 con 0x7f600c0051c0 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f602628f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60200a0040 0x7f60200a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f602628f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60200a0040 0x7f60200a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40974/0 (socket says 172.21.15.50:40974) 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f602628f640 1 -- 172.21.15.50:0/4167951694 learned_addr learned my addr 172.21.15.50:0/4167951694 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f6025a8e640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0051c0 0x7f602015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.085+0000 7f602528d640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f602009dfe0 0x7f602009e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f602628f640 1 -- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0051c0 msgr2=0x7f602015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f602628f640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0051c0 0x7f602015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f602628f640 1 -- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f602009dfe0 msgr2=0x7f602009e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f602628f640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f602009dfe0 0x7f602009e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f602628f640 1 -- 172.21.15.50:0/4167951694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602015cd60 con 0x7f60200a0040 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f602628f640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60200a0040 0x7f60200a04c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6018001f70 tx=0x7f60180014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f60167fc640 1 -- 172.21.15.50:0/4167951694 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018007db0 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f60167fc640 1 -- 172.21.15.50:0/4167951694 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6018006d60 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60201f4fd0 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f60167fc640 1 -- 172.21.15.50:0/4167951694 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018039680 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.086+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60200a10a0 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.087+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60200756f0 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.087+0000 7f60167fc640 1 -- 172.21.15.50:0/4167951694 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6018039820 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.088+0000 7f60167fc640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f600406c8f0 0x7f600406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.088+0000 7f60167fc640 1 -- 172.21.15.50:0/4167951694 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f60180ad850 con 0x7f60200a0040 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.088+0000 7f6025a8e640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f600406c8f0 0x7f600406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.089+0000 7f6025a8e640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f600406c8f0 0x7f600406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f601c001ba0 tx=0x7f601c001b30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.092+0000 7f60167fc640 1 -- 172.21.15.50:0/4167951694 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f602015cd60 con 0x7f60200a0040 2024-04-02T16:23:19.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.279+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6020161150 con 0x7f60200a0040 2024-04-02T16:23:19.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.280+0000 7f60167fc640 1 -- 172.21.15.50:0/4167951694 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16930 (secure 0 0 0) 0x7f601803e070 con 0x7f60200a0040 2024-04-02T16:23:19.281 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:19.281 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":39,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T16:23:17.582407+0000","last_up_change":"2024-04-02T16:21:47.077694+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:23:13.826831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-02T16:23:14.911147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists","up"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists","up"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.282+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f600406c8f0 msgr2=0x7f600406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.282+0000 7f6027cf2640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f600406c8f0 0x7f600406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f601c001ba0 tx=0x7f601c001b30 comp rx=0 tx=0).stop 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.282+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60200a0040 msgr2=0x7f60200a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.282+0000 7f6027cf2640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60200a0040 0x7f60200a04c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6018001f70 tx=0x7f60180014a0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 shutdown_connections 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f600406c8f0 0x7f600406edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60200a0040 0x7f60200a04c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f602009dfe0 0x7f602009e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 --2- 172.21.15.50:0/4167951694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0051c0 0x7f602015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 >> 172.21.15.50:0/4167951694 conn(0x7f602009d570 msgr2=0x7f602015f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:19.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 shutdown_connections 2024-04-02T16:23:19.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.283+0000 7f6027cf2640 1 -- 172.21.15.50:0/4167951694 wait complete. 2024-04-02T16:23:19.294 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T16:23:19.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:23:19.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.396+0000 7fe920715640 1 Processor -- start 2024-04-02T16:23:19.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.396+0000 7fe920715640 1 -- start start 2024-04-02T16:23:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe920715640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe91813deb0 0x7fe9180b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe920715640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe91815bfe0 0x7fe9180ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe920715640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe91815faf0 con 0x7fe9180b2100 2024-04-02T16:23:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe920715640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7fe91815c410 con 0x7fe91813deb0 2024-04-02T16:23:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe920715640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe91815ed30 con 0x7fe91815bfe0 2024-04-02T16:23:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91e4b1640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7fe91813deb0 0x7fe9180b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91dcb0640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fe9180b2100 0x7fe91815b8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42464/0 (socket says 172.21.15.50:42464) 2024-04-02T16:23:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91dcb0640 1 -- 172.21.15.50:0/3488144825 learned_addr learned my addr 172.21.15.50:0/3488144825 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91ecb2640 1 --2- 172.21.15.50:0/3488144825 >> v2:172.21.15.188:3300/0 conn(0x7fe91815bfe0 0x7fe9180ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91e4b1640 1 -- 172.21.15.50:0/3488144825 >> v2:172.21.15.188:3300/0 conn(0x7fe91815bfe0 msgr2=0x7fe9180ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91e4b1640 1 --2- 172.21.15.50:0/3488144825 >> v2:172.21.15.188:3300/0 conn(0x7fe91815bfe0 0x7fe9180ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91e4b1640 1 -- 172.21.15.50:0/3488144825 >> v1:172.21.15.50:6789/0 conn(0x7fe9180b2100 legacy=0x7fe91815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91e4b1640 1 -- 172.21.15.50:0/3488144825 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91815f010 con 0x7fe91813deb0 2024-04-02T16:23:19.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.397+0000 7fe91d4af640 1 -- 172.21.15.50:0/3488144825 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310733625 0 0) 0x7fe91815faf0 con 0x7fe9180b2100 2024-04-02T16:23:19.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.398+0000 7fe91e4b1640 1 --2- 172.21.15.50:0/3488144825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91813deb0 0x7fe9180b1bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe91806f520 tx=0x7fe91402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d139dfb19835e4e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.398+0000 7fe91d4af640 1 -- 172.21.15.50:0/3488144825 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914035020 con 0x7fe91813deb0 2024-04-02T16:23:19.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.398+0000 7fe91d4af640 1 -- 172.21.15.50:0/3488144825 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe914031830 con 0x7fe91813deb0 2024-04-02T16:23:19.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.398+0000 7fe91d4af640 1 -- 172.21.15.50:0/3488144825 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914031c00 con 0x7fe91813deb0 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 -- 172.21.15.50:0/3488144825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91813deb0 msgr2=0x7fe9180b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 --2- 172.21.15.50:0/3488144825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91813deb0 0x7fe9180b1bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe91806f520 tx=0x7fe91402b8d0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 -- 172.21.15.50:0/3488144825 shutdown_connections 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 --2- 172.21.15.50:0/3488144825 >> v2:172.21.15.188:3300/0 conn(0x7fe91815bfe0 0x7fe9180ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 --2- 172.21.15.50:0/3488144825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91813deb0 0x7fe9180b1bc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 -- 172.21.15.50:0/3488144825 >> 172.21.15.50:0/3488144825 conn(0x7fe91806ebf0 msgr2=0x7fe91806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 -- 172.21.15.50:0/3488144825 shutdown_connections 2024-04-02T16:23:19.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.399+0000 7fe920715640 1 -- 172.21.15.50:0/3488144825 wait complete. 2024-04-02T16:23:19.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.400+0000 7fe920715640 1 Processor -- start 2024-04-02T16:23:19.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.400+0000 7fe920715640 1 -- start start 2024-04-02T16:23:19.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180b2100 0x7fe91806d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91813deb0 0x7fe9181535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91815bfe0 0x7fe918153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe91813e290 con 0x7fe91815bfe0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe91815f5d0 con 0x7fe91813deb0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe91813e410 con 0x7fe9180b2100 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91815bfe0 0x7fe918153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91815bfe0 0x7fe918153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40984/0 (socket says 172.21.15.50:40984) 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 -- 172.21.15.50:0/2308931680 learned_addr learned my addr 172.21.15.50:0/2308931680 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 -- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180b2100 msgr2=0x7fe91806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180b2100 0x7fe91806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 -- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91813deb0 msgr2=0x7fe9181535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91813deb0 0x7fe9181535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 -- 172.21.15.50:0/2308931680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe918159880 con 0x7fe91815bfe0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe91ecb2640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91815bfe0 0x7fe918153b00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe91000cc60 tx=0x7fe91000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe90effd640 1 -- 172.21.15.50:0/2308931680 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910003990 con 0x7fe91815bfe0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe90effd640 1 -- 172.21.15.50:0/2308931680 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe910003b30 con 0x7fe91815bfe0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe918159b60 con 0x7fe91815bfe0 2024-04-02T16:23:19.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe90effd640 1 -- 172.21.15.50:0/2308931680 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910004020 con 0x7fe91815bfe0 2024-04-02T16:23:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.401+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91815a120 con 0x7fe91815bfe0 2024-04-02T16:23:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.402+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9180756f0 con 0x7fe91815bfe0 2024-04-02T16:23:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.403+0000 7fe90effd640 1 -- 172.21.15.50:0/2308931680 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9100041c0 con 0x7fe91815bfe0 2024-04-02T16:23:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.403+0000 7fe90effd640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe8fc06c8f0 0x7fe8fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.403+0000 7fe90effd640 1 -- 172.21.15.50:0/2308931680 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe91008c1d0 con 0x7fe91815bfe0 2024-04-02T16:23:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.403+0000 7fe91e4b1640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe8fc06c8f0 0x7fe8fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.404+0000 7fe91e4b1640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe8fc06c8f0 0x7fe8fc06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe914036000 tx=0x7fe914030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.407+0000 7fe90effd640 1 -- 172.21.15.50:0/2308931680 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe910014030 con 0x7fe91815bfe0 2024-04-02T16:23:19.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.588+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 --> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe91806c210 con 0x7fe8fc06c8f0 2024-04-02T16:23:19.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.594+0000 7fe90effd640 1 -- 172.21.15.50:0/2308931680 <== mgr.4112 v2:172.21.15.188:6832/2785600878 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7fe91806c210 con 0x7fe8fc06c8f0 2024-04-02T16:23:19.598 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:19.599 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe8fc06c8f0 msgr2=0x7fe8fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe8fc06c8f0 0x7fe8fc06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe914036000 tx=0x7fe914030000 comp rx=0 tx=0).stop 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91815bfe0 msgr2=0x7fe918153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91815bfe0 0x7fe918153b00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe91000cc60 tx=0x7fe91000a5c0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 shutdown_connections 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fe8fc06c8f0 0x7fe8fc06edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe91815bfe0 0x7fe918153b00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91813deb0 0x7fe9181535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 --2- 172.21.15.50:0/2308931680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180b2100 0x7fe91806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.597+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 >> 172.21.15.50:0/2308931680 conn(0x7fe91806ebf0 msgr2=0x7fe91815b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:19.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.598+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 shutdown_connections 2024-04-02T16:23:19.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.598+0000 7fe920715640 1 -- 172.21.15.50:0/2308931680 wait complete. 2024-04-02T16:23:19.608 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-02T16:23:18.684617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237472,"kb_used_data":4660,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737248,"statfs":{"total":773094113280,"available":772850941952,"internally_reserved":0,"allocated":4771840,"data_stored":2717392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.110661"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586188+0000","last_change":"2024-04-02T16:23:16.610917+0000","last_active":"2024-04-02T16:23:17.586188+0000","last_peered":"2024-04-02T16:23:17.586188+0000","last_clean":"2024-04-02T16:23:17.586188+0000","last_became_active":"2024-04-02T16:23:16.610588+0000","last_became_peered":"2024-04-02T16:23:16.610588+0000","last_unstale":"2024-04-02T16:23:17.586188+0000","last_undegraded":"2024-04-02T16:23:17.586188+0000","last_fullsized":"2024-04-02T16:23:17.586188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:30:47.908172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586255+0000","last_change":"2024-04-02T16:23:16.610552+0000","last_active":"2024-04-02T16:23:17.586255+0000","last_peered":"2024-04-02T16:23:17.586255+0000","last_clean":"2024-04-02T16:23:17.586255+0000","last_became_active":"2024-04-02T16:23:16.604220+0000","last_became_peered":"2024-04-02T16:23:16.604220+0000","last_unstale":"2024-04-02T16:23:17.586255+0000","last_undegraded":"2024-04-02T16:23:17.586255+0000","last_fullsized":"2024-04-02T16:23:17.586255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:36:28.419284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.611086+0000","last_change":"2024-04-02T16:23:16.606945+0000","last_active":"2024-04-02T16:23:16.611086+0000","last_peered":"2024-04-02T16:23:16.611086+0000","last_clean":"2024-04-02T16:23:16.611086+0000","last_became_active":"2024-04-02T16:23:16.606617+0000","last_became_peered":"2024-04-02T16:23:16.606617+0000","last_unstale":"2024-04-02T16:23:16.611086+0000","last_undegraded":"2024-04-02T16:23:16.611086+0000","last_fullsized":"2024-04-02T16:23:16.611086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:23.481064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605861+0000","last_change":"2024-04-02T16:23:16.603972+0000","last_active":"2024-04-02T16:23:17.605861+0000","last_peered":"2024-04-02T16:23:17.605861+0000","last_clean":"2024-04-02T16:23:17.605861+0000","last_became_active":"2024-04-02T16:23:16.602669+0000","last_became_peered":"2024-04-02T16:23:16.602669+0000","last_unstale":"2024-04-02T16:23:17.605861+0000","last_undegraded":"2024-04-02T16:23:17.605861+0000","last_fullsized":"2024-04-02T16:23:17.605861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:10.831124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605298+0000","last_change":"2024-04-02T16:23:16.591590+0000","last_active":"2024-04-02T16:23:17.605298+0000","last_peered":"2024-04-02T16:23:17.605298+0000","last_clean":"2024-04-02T16:23:17.605298+0000","last_became_active":"2024-04-02T16:23:16.591314+0000","last_became_peered":"2024-04-02T16:23:16.591314+0000","last_unstale":"2024-04-02T16:23:17.605298+0000","last_undegraded":"2024-04-02T16:23:17.605298+0000","last_fullsized":"2024-04-02T16:23:17.605298+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:59.183875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.600476+0000","last_change":"2024-04-02T16:23:16.596994+0000","last_active":"2024-04-02T16:23:16.600476+0000","last_peered":"2024-04-02T16:23:16.600476+0000","last_clean":"2024-04-02T16:23:16.600476+0000","last_became_active":"2024-04-02T16:23:16.596484+0000","last_became_peered":"2024-04-02T16:23:16.596484+0000","last_unstale":"2024-04-02T16:23:16.600476+0000","last_undegraded":"2024-04-02T16:23:16.600476+0000","last_fullsized":"2024-04-02T16:23:16.600476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:59.966073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586860+0000","last_change":"2024-04-02T16:23:16.598667+0000","last_active":"2024-04-02T16:23:17.586860+0000","last_peered":"2024-04-02T16:23:17.586860+0000","last_clean":"2024-04-02T16:23:17.586860+0000","last_became_active":"2024-04-02T16:23:16.598394+0000","last_became_peered":"2024-04-02T16:23:16.598394+0000","last_unstale":"2024-04-02T16:23:17.586860+0000","last_undegraded":"2024-04-02T16:23:17.586860+0000","last_fullsized":"2024-04-02T16:23:17.586860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:50.721161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585736+0000","last_change":"2024-04-02T16:23:16.610689+0000","last_active":"2024-04-02T16:23:17.585736+0000","last_peered":"2024-04-02T16:23:17.585736+0000","last_clean":"2024-04-02T16:23:17.585736+0000","last_became_active":"2024-04-02T16:23:16.605655+0000","last_became_peered":"2024-04-02T16:23:16.605655+0000","last_unstale":"2024-04-02T16:23:17.585736+0000","last_undegraded":"2024-04-02T16:23:17.585736+0000","last_fullsized":"2024-04-02T16:23:17.585736+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:17:49.482366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610813+0000","last_change":"2024-04-02T16:23:16.606933+0000","last_active":"2024-04-02T16:23:16.610813+0000","last_peered":"2024-04-02T16:23:16.610813+0000","last_clean":"2024-04-02T16:23:16.610813+0000","last_became_active":"2024-04-02T16:23:16.606627+0000","last_became_peered":"2024-04-02T16:23:16.606627+0000","last_unstale":"2024-04-02T16:23:16.610813+0000","last_undegraded":"2024-04-02T16:23:16.610813+0000","last_fullsized":"2024-04-02T16:23:16.610813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:52.688258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587186+0000","last_change":"2024-04-02T16:23:16.607054+0000","last_active":"2024-04-02T16:23:17.587186+0000","last_peered":"2024-04-02T16:23:17.587186+0000","last_clean":"2024-04-02T16:23:17.587186+0000","last_became_active":"2024-04-02T16:23:16.606207+0000","last_became_peered":"2024-04-02T16:23:16.606207+0000","last_unstale":"2024-04-02T16:23:17.587186+0000","last_undegraded":"2024-04-02T16:23:17.587186+0000","last_fullsized":"2024-04-02T16:23:17.587186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:33:49.551083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586799+0000","last_change":"2024-04-02T16:23:16.603903+0000","last_active":"2024-04-02T16:23:17.586799+0000","last_peered":"2024-04-02T16:23:17.586799+0000","last_clean":"2024-04-02T16:23:17.586799+0000","last_became_active":"2024-04-02T16:23:16.603621+0000","last_became_peered":"2024-04-02T16:23:16.603621+0000","last_unstale":"2024-04-02T16:23:17.586799+0000","last_undegraded":"2024-04-02T16:23:17.586799+0000","last_fullsized":"2024-04-02T16:23:17.586799+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:59.438279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610804+0000","last_change":"2024-04-02T16:23:16.607184+0000","last_active":"2024-04-02T16:23:16.610804+0000","last_peered":"2024-04-02T16:23:16.610804+0000","last_clean":"2024-04-02T16:23:16.610804+0000","last_became_active":"2024-04-02T16:23:16.606669+0000","last_became_peered":"2024-04-02T16:23:16.606669+0000","last_unstale":"2024-04-02T16:23:16.610804+0000","last_undegraded":"2024-04-02T16:23:16.610804+0000","last_fullsized":"2024-04-02T16:23:16.610804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:13.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586801+0000","last_change":"2024-04-02T16:23:16.605049+0000","last_active":"2024-04-02T16:23:17.586801+0000","last_peered":"2024-04-02T16:23:17.586801+0000","last_clean":"2024-04-02T16:23:17.586801+0000","last_became_active":"2024-04-02T16:23:16.604743+0000","last_became_peered":"2024-04-02T16:23:16.604743+0000","last_unstale":"2024-04-02T16:23:17.586801+0000","last_undegraded":"2024-04-02T16:23:17.586801+0000","last_fullsized":"2024-04-02T16:23:17.586801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:24.435217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587474+0000","last_change":"2024-04-02T16:23:16.600887+0000","last_active":"2024-04-02T16:23:17.587474+0000","last_peered":"2024-04-02T16:23:17.587474+0000","last_clean":"2024-04-02T16:23:17.587474+0000","last_became_active":"2024-04-02T16:23:16.600585+0000","last_became_peered":"2024-04-02T16:23:16.600585+0000","last_unstale":"2024-04-02T16:23:17.587474+0000","last_undegraded":"2024-04-02T16:23:17.587474+0000","last_fullsized":"2024-04-02T16:23:17.587474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:38.558835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587238+0000","last_change":"2024-04-02T16:23:16.598969+0000","last_active":"2024-04-02T16:23:17.587238+0000","last_peered":"2024-04-02T16:23:17.587238+0000","last_clean":"2024-04-02T16:23:17.587238+0000","last_became_active":"2024-04-02T16:23:16.597796+0000","last_became_peered":"2024-04-02T16:23:16.597796+0000","last_unstale":"2024-04-02T16:23:17.587238+0000","last_undegraded":"2024-04-02T16:23:17.587238+0000","last_fullsized":"2024-04-02T16:23:17.587238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:00.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.607772+0000","last_change":"2024-04-02T16:23:16.601475+0000","last_active":"2024-04-02T16:23:16.607772+0000","last_peered":"2024-04-02T16:23:16.607772+0000","last_clean":"2024-04-02T16:23:16.607772+0000","last_became_active":"2024-04-02T16:23:16.601181+0000","last_became_peered":"2024-04-02T16:23:16.601181+0000","last_unstale":"2024-04-02T16:23:16.607772+0000","last_undegraded":"2024-04-02T16:23:16.607772+0000","last_fullsized":"2024-04-02T16:23:16.607772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:23.481064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.607780+0000","last_change":"2024-04-02T16:23:16.602426+0000","last_active":"2024-04-02T16:23:16.607780+0000","last_peered":"2024-04-02T16:23:16.607780+0000","last_clean":"2024-04-02T16:23:16.607780+0000","last_became_active":"2024-04-02T16:23:16.602147+0000","last_became_peered":"2024-04-02T16:23:16.602147+0000","last_unstale":"2024-04-02T16:23:16.607780+0000","last_undegraded":"2024-04-02T16:23:16.607780+0000","last_fullsized":"2024-04-02T16:23:16.607780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:46.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.613581+0000","last_change":"2024-04-02T16:23:16.600782+0000","last_active":"2024-04-02T16:23:16.613581+0000","last_peered":"2024-04-02T16:23:16.613581+0000","last_clean":"2024-04-02T16:23:16.613581+0000","last_became_active":"2024-04-02T16:23:16.600521+0000","last_became_peered":"2024-04-02T16:23:16.600521+0000","last_unstale":"2024-04-02T16:23:16.613581+0000","last_undegraded":"2024-04-02T16:23:16.613581+0000","last_fullsized":"2024-04-02T16:23:16.613581+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:13.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605379+0000","last_change":"2024-04-02T16:23:16.604296+0000","last_active":"2024-04-02T16:23:17.605379+0000","last_peered":"2024-04-02T16:23:17.605379+0000","last_clean":"2024-04-02T16:23:17.605379+0000","last_became_active":"2024-04-02T16:23:16.603883+0000","last_became_peered":"2024-04-02T16:23:16.603883+0000","last_unstale":"2024-04-02T16:23:17.605379+0000","last_undegraded":"2024-04-02T16:23:17.605379+0000","last_fullsized":"2024-04-02T16:23:17.605379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:47.341504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.596264+0000","last_change":"2024-04-02T16:23:16.590853+0000","last_active":"2024-04-02T16:23:16.596264+0000","last_peered":"2024-04-02T16:23:16.596264+0000","last_clean":"2024-04-02T16:23:16.596264+0000","last_became_active":"2024-04-02T16:23:16.590574+0000","last_became_peered":"2024-04-02T16:23:16.590574+0000","last_unstale":"2024-04-02T16:23:16.596264+0000","last_undegraded":"2024-04-02T16:23:16.596264+0000","last_fullsized":"2024-04-02T16:23:16.596264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:47.341504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586985+0000","last_change":"2024-04-02T16:23:16.599240+0000","last_active":"2024-04-02T16:23:17.586985+0000","last_peered":"2024-04-02T16:23:17.586985+0000","last_clean":"2024-04-02T16:23:17.586985+0000","last_became_active":"2024-04-02T16:23:16.598954+0000","last_became_peered":"2024-04-02T16:23:16.598954+0000","last_unstale":"2024-04-02T16:23:17.586985+0000","last_undegraded":"2024-04-02T16:23:17.586985+0000","last_fullsized":"2024-04-02T16:23:17.586985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:17:49.482366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587828+0000","last_change":"2024-04-02T16:23:16.594822+0000","last_active":"2024-04-02T16:23:17.587828+0000","last_peered":"2024-04-02T16:23:17.587828+0000","last_clean":"2024-04-02T16:23:17.587828+0000","last_became_active":"2024-04-02T16:23:16.594534+0000","last_became_peered":"2024-04-02T16:23:16.594534+0000","last_unstale":"2024-04-02T16:23:17.587828+0000","last_undegraded":"2024-04-02T16:23:17.587828+0000","last_fullsized":"2024-04-02T16:23:17.587828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:24.435217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610368+0000","last_change":"2024-04-02T16:23:16.604204+0000","last_active":"2024-04-02T16:23:16.610368+0000","last_peered":"2024-04-02T16:23:16.610368+0000","last_clean":"2024-04-02T16:23:16.610368+0000","last_became_active":"2024-04-02T16:23:16.603946+0000","last_became_peered":"2024-04-02T16:23:16.603946+0000","last_unstale":"2024-04-02T16:23:16.610368+0000","last_undegraded":"2024-04-02T16:23:16.610368+0000","last_fullsized":"2024-04-02T16:23:16.610368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:17.861607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605278+0000","last_change":"2024-04-02T16:23:16.603268+0000","last_active":"2024-04-02T16:23:17.605278+0000","last_peered":"2024-04-02T16:23:17.605278+0000","last_clean":"2024-04-02T16:23:17.605278+0000","last_became_active":"2024-04-02T16:23:16.602949+0000","last_became_peered":"2024-04-02T16:23:16.602949+0000","last_unstale":"2024-04-02T16:23:17.605278+0000","last_undegraded":"2024-04-02T16:23:17.605278+0000","last_fullsized":"2024-04-02T16:23:17.605278+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:24.522597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.615529+0000","last_change":"2024-04-02T16:23:16.611562+0000","last_active":"2024-04-02T16:23:16.615529+0000","last_peered":"2024-04-02T16:23:16.615529+0000","last_clean":"2024-04-02T16:23:16.615529+0000","last_became_active":"2024-04-02T16:23:16.611263+0000","last_became_peered":"2024-04-02T16:23:16.611263+0000","last_unstale":"2024-04-02T16:23:16.615529+0000","last_undegraded":"2024-04-02T16:23:16.615529+0000","last_fullsized":"2024-04-02T16:23:16.615529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:44.379025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610323+0000","last_change":"2024-04-02T16:23:16.606339+0000","last_active":"2024-04-02T16:23:16.610323+0000","last_peered":"2024-04-02T16:23:16.610323+0000","last_clean":"2024-04-02T16:23:16.610323+0000","last_became_active":"2024-04-02T16:23:16.604419+0000","last_became_peered":"2024-04-02T16:23:16.604419+0000","last_unstale":"2024-04-02T16:23:16.610323+0000","last_undegraded":"2024-04-02T16:23:16.610323+0000","last_fullsized":"2024-04-02T16:23:16.610323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:57.683621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.597583+0000","last_change":"2024-04-02T16:23:16.593111+0000","last_active":"2024-04-02T16:23:16.597583+0000","last_peered":"2024-04-02T16:23:16.597583+0000","last_clean":"2024-04-02T16:23:16.597583+0000","last_became_active":"2024-04-02T16:23:16.592827+0000","last_became_peered":"2024-04-02T16:23:16.592827+0000","last_unstale":"2024-04-02T16:23:16.597583+0000","last_undegraded":"2024-04-02T16:23:16.597583+0000","last_fullsized":"2024-04-02T16:23:16.597583+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:10.831124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587582+0000","last_change":"2024-04-02T16:23:16.605966+0000","last_active":"2024-04-02T16:23:17.587582+0000","last_peered":"2024-04-02T16:23:17.587582+0000","last_clean":"2024-04-02T16:23:17.587582+0000","last_became_active":"2024-04-02T16:23:16.605495+0000","last_became_peered":"2024-04-02T16:23:16.605495+0000","last_unstale":"2024-04-02T16:23:17.587582+0000","last_undegraded":"2024-04-02T16:23:17.587582+0000","last_fullsized":"2024-04-02T16:23:17.587582+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:30:47.908172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610366+0000","last_change":"2024-04-02T16:23:16.597164+0000","last_active":"2024-04-02T16:23:16.610366+0000","last_peered":"2024-04-02T16:23:16.610366+0000","last_clean":"2024-04-02T16:23:16.610366+0000","last_became_active":"2024-04-02T16:23:16.596897+0000","last_became_peered":"2024-04-02T16:23:16.596897+0000","last_unstale":"2024-04-02T16:23:16.610366+0000","last_undegraded":"2024-04-02T16:23:16.610366+0000","last_fullsized":"2024-04-02T16:23:16.610366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:52.688258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610141+0000","last_change":"2024-04-02T16:23:16.596256+0000","last_active":"2024-04-02T16:23:16.610141+0000","last_peered":"2024-04-02T16:23:16.610141+0000","last_clean":"2024-04-02T16:23:16.610141+0000","last_became_active":"2024-04-02T16:23:16.595967+0000","last_became_peered":"2024-04-02T16:23:16.595967+0000","last_unstale":"2024-04-02T16:23:16.610141+0000","last_undegraded":"2024-04-02T16:23:16.610141+0000","last_fullsized":"2024-04-02T16:23:16.610141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:45.384355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610738+0000","last_change":"2024-04-02T16:23:16.604797+0000","last_active":"2024-04-02T16:23:16.610738+0000","last_peered":"2024-04-02T16:23:16.610738+0000","last_clean":"2024-04-02T16:23:16.610738+0000","last_became_active":"2024-04-02T16:23:16.604507+0000","last_became_peered":"2024-04-02T16:23:16.604507+0000","last_unstale":"2024-04-02T16:23:16.610738+0000","last_undegraded":"2024-04-02T16:23:16.610738+0000","last_fullsized":"2024-04-02T16:23:16.610738+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:26.024041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610898+0000","last_change":"2024-04-02T16:23:16.605620+0000","last_active":"2024-04-02T16:23:16.610898+0000","last_peered":"2024-04-02T16:23:16.610898+0000","last_clean":"2024-04-02T16:23:16.610898+0000","last_became_active":"2024-04-02T16:23:16.605316+0000","last_became_peered":"2024-04-02T16:23:16.605316+0000","last_unstale":"2024-04-02T16:23:16.610898+0000","last_undegraded":"2024-04-02T16:23:16.610898+0000","last_fullsized":"2024-04-02T16:23:16.610898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:26.024041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.607746+0000","last_change":"2024-04-02T16:23:16.601983+0000","last_active":"2024-04-02T16:23:16.607746+0000","last_peered":"2024-04-02T16:23:16.607746+0000","last_clean":"2024-04-02T16:23:16.607746+0000","last_became_active":"2024-04-02T16:23:16.601735+0000","last_became_peered":"2024-04-02T16:23:16.601735+0000","last_unstale":"2024-04-02T16:23:16.607746+0000","last_undegraded":"2024-04-02T16:23:16.607746+0000","last_fullsized":"2024-04-02T16:23:16.607746+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:59.966073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.618649+0000","last_change":"2024-04-02T16:23:15.599944+0000","last_active":"2024-04-02T16:23:16.618649+0000","last_peered":"2024-04-02T16:23:16.618649+0000","last_clean":"2024-04-02T16:23:16.618649+0000","last_became_active":"2024-04-02T16:23:15.599613+0000","last_became_peered":"2024-04-02T16:23:15.599613+0000","last_unstale":"2024-04-02T16:23:16.618649+0000","last_undegraded":"2024-04-02T16:23:16.618649+0000","last_fullsized":"2024-04-02T16:23:16.618649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:23.520265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.612177+0000","last_change":"2024-04-02T16:23:16.606292+0000","last_active":"2024-04-02T16:23:16.612177+0000","last_peered":"2024-04-02T16:23:16.612177+0000","last_clean":"2024-04-02T16:23:16.612177+0000","last_became_active":"2024-04-02T16:23:16.604827+0000","last_became_peered":"2024-04-02T16:23:16.604827+0000","last_unstale":"2024-04-02T16:23:16.612177+0000","last_undegraded":"2024-04-02T16:23:16.612177+0000","last_fullsized":"2024-04-02T16:23:16.612177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:57:04.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.608260+0000","last_change":"2024-04-02T16:23:15.599256+0000","last_active":"2024-04-02T16:23:16.608260+0000","last_peered":"2024-04-02T16:23:16.608260+0000","last_clean":"2024-04-02T16:23:16.608260+0000","last_became_active":"2024-04-02T16:23:15.598974+0000","last_became_peered":"2024-04-02T16:23:15.598974+0000","last_unstale":"2024-04-02T16:23:16.608260+0000","last_undegraded":"2024-04-02T16:23:16.608260+0000","last_fullsized":"2024-04-02T16:23:16.608260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:09.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587777+0000","last_change":"2024-04-02T16:23:16.600799+0000","last_active":"2024-04-02T16:23:17.587777+0000","last_peered":"2024-04-02T16:23:17.587777+0000","last_clean":"2024-04-02T16:23:17.587777+0000","last_became_active":"2024-04-02T16:23:16.600154+0000","last_became_peered":"2024-04-02T16:23:16.600154+0000","last_unstale":"2024-04-02T16:23:17.587777+0000","last_undegraded":"2024-04-02T16:23:17.587777+0000","last_fullsized":"2024-04-02T16:23:17.587777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:39.120379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.618456+0000","last_change":"2024-04-02T16:23:15.598575+0000","last_active":"2024-04-02T16:23:16.618456+0000","last_peered":"2024-04-02T16:23:16.618456+0000","last_clean":"2024-04-02T16:23:16.618456+0000","last_became_active":"2024-04-02T16:23:15.598268+0000","last_became_peered":"2024-04-02T16:23:15.598268+0000","last_unstale":"2024-04-02T16:23:16.618456+0000","last_undegraded":"2024-04-02T16:23:16.618456+0000","last_fullsized":"2024-04-02T16:23:16.618456+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:22.478731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586245+0000","last_change":"2024-04-02T16:23:16.598704+0000","last_active":"2024-04-02T16:23:17.586245+0000","last_peered":"2024-04-02T16:23:17.586245+0000","last_clean":"2024-04-02T16:23:17.586245+0000","last_became_active":"2024-04-02T16:23:16.598449+0000","last_became_peered":"2024-04-02T16:23:16.598449+0000","last_unstale":"2024-04-02T16:23:17.586245+0000","last_undegraded":"2024-04-02T16:23:17.586245+0000","last_fullsized":"2024-04-02T16:23:17.586245+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:40.190923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.588148+0000","last_change":"2024-04-02T16:23:15.596983+0000","last_active":"2024-04-02T16:23:17.588148+0000","last_peered":"2024-04-02T16:23:17.588148+0000","last_clean":"2024-04-02T16:23:17.588148+0000","last_became_active":"2024-04-02T16:23:15.596705+0000","last_became_peered":"2024-04-02T16:23:15.596705+0000","last_unstale":"2024-04-02T16:23:17.588148+0000","last_undegraded":"2024-04-02T16:23:17.588148+0000","last_fullsized":"2024-04-02T16:23:17.588148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:12.035337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.614269+0000","last_change":"2024-04-02T16:23:16.608230+0000","last_active":"2024-04-02T16:23:16.614269+0000","last_peered":"2024-04-02T16:23:16.614269+0000","last_clean":"2024-04-02T16:23:16.614269+0000","last_became_active":"2024-04-02T16:23:16.607980+0000","last_became_peered":"2024-04-02T16:23:16.607980+0000","last_unstale":"2024-04-02T16:23:16.614269+0000","last_undegraded":"2024-04-02T16:23:16.614269+0000","last_fullsized":"2024-04-02T16:23:16.614269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:17.861607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587103+0000","last_change":"2024-04-02T16:23:15.601122+0000","last_active":"2024-04-02T16:23:17.587103+0000","last_peered":"2024-04-02T16:23:17.587103+0000","last_clean":"2024-04-02T16:23:17.587103+0000","last_became_active":"2024-04-02T16:23:15.600811+0000","last_became_peered":"2024-04-02T16:23:15.600811+0000","last_unstale":"2024-04-02T16:23:17.587103+0000","last_undegraded":"2024-04-02T16:23:17.587103+0000","last_fullsized":"2024-04-02T16:23:17.587103+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:22.478731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587143+0000","last_change":"2024-04-02T16:23:16.594813+0000","last_active":"2024-04-02T16:23:17.587143+0000","last_peered":"2024-04-02T16:23:17.587143+0000","last_clean":"2024-04-02T16:23:17.587143+0000","last_became_active":"2024-04-02T16:23:16.594564+0000","last_became_peered":"2024-04-02T16:23:16.594564+0000","last_unstale":"2024-04-02T16:23:17.587143+0000","last_undegraded":"2024-04-02T16:23:17.587143+0000","last_fullsized":"2024-04-02T16:23:17.587143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:51.357191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.608953+0000","last_change":"2024-04-02T16:23:15.599786+0000","last_active":"2024-04-02T16:23:16.608953+0000","last_peered":"2024-04-02T16:23:16.608953+0000","last_clean":"2024-04-02T16:23:16.608953+0000","last_became_active":"2024-04-02T16:23:15.599262+0000","last_became_peered":"2024-04-02T16:23:15.599262+0000","last_unstale":"2024-04-02T16:23:16.608953+0000","last_undegraded":"2024-04-02T16:23:16.608953+0000","last_fullsized":"2024-04-02T16:23:16.608953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:54.455581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587838+0000","last_change":"2024-04-02T16:23:16.602963+0000","last_active":"2024-04-02T16:23:17.587838+0000","last_peered":"2024-04-02T16:23:17.587838+0000","last_clean":"2024-04-02T16:23:17.587838+0000","last_became_active":"2024-04-02T16:23:16.602692+0000","last_became_peered":"2024-04-02T16:23:16.602692+0000","last_unstale":"2024-04-02T16:23:17.587838+0000","last_undegraded":"2024-04-02T16:23:17.587838+0000","last_fullsized":"2024-04-02T16:23:17.587838+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:59.438279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.590093+0000","last_change":"2024-04-02T16:23:15.593798+0000","last_active":"2024-04-02T16:23:16.590093+0000","last_peered":"2024-04-02T16:23:16.590093+0000","last_clean":"2024-04-02T16:23:16.590093+0000","last_became_active":"2024-04-02T16:23:15.593534+0000","last_became_peered":"2024-04-02T16:23:15.593534+0000","last_unstale":"2024-04-02T16:23:16.590093+0000","last_undegraded":"2024-04-02T16:23:16.590093+0000","last_fullsized":"2024-04-02T16:23:16.590093+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:51.685925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585983+0000","last_change":"2024-04-02T16:23:16.599801+0000","last_active":"2024-04-02T16:23:17.585983+0000","last_peered":"2024-04-02T16:23:17.585983+0000","last_clean":"2024-04-02T16:23:17.585983+0000","last_became_active":"2024-04-02T16:23:16.599495+0000","last_became_peered":"2024-04-02T16:23:16.599495+0000","last_unstale":"2024-04-02T16:23:17.585983+0000","last_undegraded":"2024-04-02T16:23:17.585983+0000","last_fullsized":"2024-04-02T16:23:17.585983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:33:49.551083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587122+0000","last_change":"2024-04-02T16:23:15.594881+0000","last_active":"2024-04-02T16:23:17.587122+0000","last_peered":"2024-04-02T16:23:17.587122+0000","last_clean":"2024-04-02T16:23:17.587122+0000","last_became_active":"2024-04-02T16:23:15.594521+0000","last_became_peered":"2024-04-02T16:23:15.594521+0000","last_unstale":"2024-04-02T16:23:17.587122+0000","last_undegraded":"2024-04-02T16:23:17.587122+0000","last_fullsized":"2024-04-02T16:23:17.587122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:51.685925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605985+0000","last_change":"2024-04-02T16:23:16.603117+0000","last_active":"2024-04-02T16:23:17.605985+0000","last_peered":"2024-04-02T16:23:17.605985+0000","last_clean":"2024-04-02T16:23:17.605985+0000","last_became_active":"2024-04-02T16:23:16.599775+0000","last_became_peered":"2024-04-02T16:23:16.599775+0000","last_unstale":"2024-04-02T16:23:17.605985+0000","last_undegraded":"2024-04-02T16:23:17.605985+0000","last_fullsized":"2024-04-02T16:23:17.605985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:45.384355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586259+0000","last_change":"2024-04-02T16:23:15.591988+0000","last_active":"2024-04-02T16:23:17.586259+0000","last_peered":"2024-04-02T16:23:17.586259+0000","last_clean":"2024-04-02T16:23:17.586259+0000","last_became_active":"2024-04-02T16:23:15.591579+0000","last_became_peered":"2024-04-02T16:23:15.591579+0000","last_unstale":"2024-04-02T16:23:17.586259+0000","last_undegraded":"2024-04-02T16:23:17.586259+0000","last_fullsized":"2024-04-02T16:23:17.586259+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:25.021708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.604234+0000","last_change":"2024-04-02T16:23:16.598657+0000","last_active":"2024-04-02T16:23:16.604234+0000","last_peered":"2024-04-02T16:23:16.604234+0000","last_clean":"2024-04-02T16:23:16.604234+0000","last_became_active":"2024-04-02T16:23:16.598411+0000","last_became_peered":"2024-04-02T16:23:16.598411+0000","last_unstale":"2024-04-02T16:23:16.604234+0000","last_undegraded":"2024-04-02T16:23:16.604234+0000","last_fullsized":"2024-04-02T16:23:16.604234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:13.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587529+0000","last_change":"2024-04-02T16:23:15.591680+0000","last_active":"2024-04-02T16:23:17.587529+0000","last_peered":"2024-04-02T16:23:17.587529+0000","last_clean":"2024-04-02T16:23:17.587529+0000","last_became_active":"2024-04-02T16:23:15.591398+0000","last_became_peered":"2024-04-02T16:23:15.591398+0000","last_unstale":"2024-04-02T16:23:17.587529+0000","last_undegraded":"2024-04-02T16:23:17.587529+0000","last_fullsized":"2024-04-02T16:23:17.587529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:12.035337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.611112+0000","last_change":"2024-04-02T16:23:16.607312+0000","last_active":"2024-04-02T16:23:16.611112+0000","last_peered":"2024-04-02T16:23:16.611112+0000","last_clean":"2024-04-02T16:23:16.611112+0000","last_became_active":"2024-04-02T16:23:16.607032+0000","last_became_peered":"2024-04-02T16:23:16.607032+0000","last_unstale":"2024-04-02T16:23:16.611112+0000","last_undegraded":"2024-04-02T16:23:16.611112+0000","last_fullsized":"2024-04-02T16:23:16.611112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:59.966073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587107+0000","last_change":"2024-04-02T16:23:15.589106+0000","last_active":"2024-04-02T16:23:17.587107+0000","last_peered":"2024-04-02T16:23:17.587107+0000","last_clean":"2024-04-02T16:23:17.587107+0000","last_became_active":"2024-04-02T16:23:15.588440+0000","last_became_peered":"2024-04-02T16:23:15.588440+0000","last_unstale":"2024-04-02T16:23:17.587107+0000","last_undegraded":"2024-04-02T16:23:17.587107+0000","last_fullsized":"2024-04-02T16:23:17.587107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:43.376692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587671+0000","last_change":"2024-04-02T16:23:16.595513+0000","last_active":"2024-04-02T16:23:17.587671+0000","last_peered":"2024-04-02T16:23:17.587671+0000","last_clean":"2024-04-02T16:23:17.587671+0000","last_became_active":"2024-04-02T16:23:16.594290+0000","last_became_peered":"2024-04-02T16:23:16.594290+0000","last_unstale":"2024-04-02T16:23:17.587671+0000","last_undegraded":"2024-04-02T16:23:17.587671+0000","last_fullsized":"2024-04-02T16:23:17.587671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:23.044403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586070+0000","last_change":"2024-04-02T16:23:15.594529+0000","last_active":"2024-04-02T16:23:17.586070+0000","last_peered":"2024-04-02T16:23:17.586070+0000","last_clean":"2024-04-02T16:23:17.586070+0000","last_became_active":"2024-04-02T16:23:15.594103+0000","last_became_peered":"2024-04-02T16:23:15.594103+0000","last_unstale":"2024-04-02T16:23:17.586070+0000","last_undegraded":"2024-04-02T16:23:17.586070+0000","last_fullsized":"2024-04-02T16:23:17.586070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:57:03.723827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.614286+0000","last_change":"2024-04-02T16:23:16.610013+0000","last_active":"2024-04-02T16:23:16.614286+0000","last_peered":"2024-04-02T16:23:16.614286+0000","last_clean":"2024-04-02T16:23:16.614286+0000","last_became_active":"2024-04-02T16:23:16.609730+0000","last_became_peered":"2024-04-02T16:23:16.609730+0000","last_unstale":"2024-04-02T16:23:16.614286+0000","last_undegraded":"2024-04-02T16:23:16.614286+0000","last_fullsized":"2024-04-02T16:23:16.614286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:26.024041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586109+0000","last_change":"2024-04-02T16:23:15.601030+0000","last_active":"2024-04-02T16:23:17.586109+0000","last_peered":"2024-04-02T16:23:17.586109+0000","last_clean":"2024-04-02T16:23:17.586109+0000","last_became_active":"2024-04-02T16:23:15.600768+0000","last_became_peered":"2024-04-02T16:23:15.600768+0000","last_unstale":"2024-04-02T16:23:17.586109+0000","last_undegraded":"2024-04-02T16:23:17.586109+0000","last_fullsized":"2024-04-02T16:23:17.586109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:02:04.853324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585791+0000","last_change":"2024-04-02T16:23:16.602319+0000","last_active":"2024-04-02T16:23:17.585791+0000","last_peered":"2024-04-02T16:23:17.585791+0000","last_clean":"2024-04-02T16:23:17.585791+0000","last_became_active":"2024-04-02T16:23:16.602062+0000","last_became_peered":"2024-04-02T16:23:16.602062+0000","last_unstale":"2024-04-02T16:23:17.585791+0000","last_undegraded":"2024-04-02T16:23:17.585791+0000","last_fullsized":"2024-04-02T16:23:17.585791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:51.357191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.590720+0000","last_change":"2024-04-02T16:23:15.594534+0000","last_active":"2024-04-02T16:23:16.590720+0000","last_peered":"2024-04-02T16:23:16.590720+0000","last_clean":"2024-04-02T16:23:16.590720+0000","last_became_active":"2024-04-02T16:23:15.594233+0000","last_became_peered":"2024-04-02T16:23:15.594233+0000","last_unstale":"2024-04-02T16:23:16.590720+0000","last_undegraded":"2024-04-02T16:23:16.590720+0000","last_fullsized":"2024-04-02T16:23:16.590720+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:12.035337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.615678+0000","last_change":"2024-04-02T16:23:16.612243+0000","last_active":"2024-04-02T16:23:16.615678+0000","last_peered":"2024-04-02T16:23:16.615678+0000","last_clean":"2024-04-02T16:23:16.615678+0000","last_became_active":"2024-04-02T16:23:16.611908+0000","last_became_peered":"2024-04-02T16:23:16.611908+0000","last_unstale":"2024-04-02T16:23:16.615678+0000","last_undegraded":"2024-04-02T16:23:16.615678+0000","last_fullsized":"2024-04-02T16:23:16.615678+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:02:05.855656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.587750+0000","last_change":"2024-04-02T16:23:15.599541+0000","last_active":"2024-04-02T16:23:16.587750+0000","last_peered":"2024-04-02T16:23:16.587750+0000","last_clean":"2024-04-02T16:23:16.587750+0000","last_became_active":"2024-04-02T16:23:15.599251+0000","last_became_peered":"2024-04-02T16:23:15.599251+0000","last_unstale":"2024-04-02T16:23:16.587750+0000","last_undegraded":"2024-04-02T16:23:16.587750+0000","last_fullsized":"2024-04-02T16:23:16.587750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:58.181543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585982+0000","last_change":"2024-04-02T16:23:16.615009+0000","last_active":"2024-04-02T16:23:17.585982+0000","last_peered":"2024-04-02T16:23:17.585982+0000","last_clean":"2024-04-02T16:23:17.585982+0000","last_became_active":"2024-04-02T16:23:16.611215+0000","last_became_peered":"2024-04-02T16:23:16.611215+0000","last_unstale":"2024-04-02T16:23:17.585982+0000","last_undegraded":"2024-04-02T16:23:17.585982+0000","last_fullsized":"2024-04-02T16:23:17.585982+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:16.140068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.609628+0000","last_change":"2024-04-02T16:23:15.599617+0000","last_active":"2024-04-02T16:23:16.609628+0000","last_peered":"2024-04-02T16:23:16.609628+0000","last_clean":"2024-04-02T16:23:16.609628+0000","last_became_active":"2024-04-02T16:23:15.599314+0000","last_became_peered":"2024-04-02T16:23:15.599314+0000","last_unstale":"2024-04-02T16:23:16.609628+0000","last_undegraded":"2024-04-02T16:23:16.609628+0000","last_fullsized":"2024-04-02T16:23:16.609628+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:09.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586109+0000","last_change":"2024-04-02T16:23:16.599855+0000","last_active":"2024-04-02T16:23:17.586109+0000","last_peered":"2024-04-02T16:23:17.586109+0000","last_clean":"2024-04-02T16:23:17.586109+0000","last_became_active":"2024-04-02T16:23:16.599580+0000","last_became_peered":"2024-04-02T16:23:16.599580+0000","last_unstale":"2024-04-02T16:23:17.586109+0000","last_undegraded":"2024-04-02T16:23:17.586109+0000","last_fullsized":"2024-04-02T16:23:17.586109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:00.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587241+0000","last_change":"2024-04-02T16:23:15.597637+0000","last_active":"2024-04-02T16:23:17.587241+0000","last_peered":"2024-04-02T16:23:17.587241+0000","last_clean":"2024-04-02T16:23:17.587241+0000","last_became_active":"2024-04-02T16:23:15.597344+0000","last_became_peered":"2024-04-02T16:23:15.597344+0000","last_unstale":"2024-04-02T16:23:17.587241+0000","last_undegraded":"2024-04-02T16:23:17.587241+0000","last_fullsized":"2024-04-02T16:23:17.587241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:58.963741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586977+0000","last_change":"2024-04-02T16:23:16.598736+0000","last_active":"2024-04-02T16:23:17.586977+0000","last_peered":"2024-04-02T16:23:17.586977+0000","last_clean":"2024-04-02T16:23:17.586977+0000","last_became_active":"2024-04-02T16:23:16.598448+0000","last_became_peered":"2024-04-02T16:23:16.598448+0000","last_unstale":"2024-04-02T16:23:17.586977+0000","last_undegraded":"2024-04-02T16:23:17.586977+0000","last_fullsized":"2024-04-02T16:23:17.586977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:50.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.588018+0000","last_change":"2024-04-02T16:23:15.594971+0000","last_active":"2024-04-02T16:23:17.588018+0000","last_peered":"2024-04-02T16:23:17.588018+0000","last_clean":"2024-04-02T16:23:17.588018+0000","last_became_active":"2024-04-02T16:23:15.594680+0000","last_became_peered":"2024-04-02T16:23:15.594680+0000","last_unstale":"2024-04-02T16:23:17.588018+0000","last_undegraded":"2024-04-02T16:23:17.588018+0000","last_fullsized":"2024-04-02T16:23:17.588018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:22.478731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587787+0000","last_change":"2024-04-02T16:23:16.600865+0000","last_active":"2024-04-02T16:23:17.587787+0000","last_peered":"2024-04-02T16:23:17.587787+0000","last_clean":"2024-04-02T16:23:17.587787+0000","last_became_active":"2024-04-02T16:23:16.600607+0000","last_became_peered":"2024-04-02T16:23:16.600607+0000","last_unstale":"2024-04-02T16:23:17.587787+0000","last_undegraded":"2024-04-02T16:23:17.587787+0000","last_fullsized":"2024-04-02T16:23:17.587787+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:54:53.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605464+0000","last_change":"2024-04-02T16:23:15.594551+0000","last_active":"2024-04-02T16:23:17.605464+0000","last_peered":"2024-04-02T16:23:17.605464+0000","last_clean":"2024-04-02T16:23:17.605464+0000","last_became_active":"2024-04-02T16:23:15.594221+0000","last_became_peered":"2024-04-02T16:23:15.594221+0000","last_unstale":"2024-04-02T16:23:17.605464+0000","last_undegraded":"2024-04-02T16:23:17.605464+0000","last_fullsized":"2024-04-02T16:23:17.605464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:50.964903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605518+0000","last_change":"2024-04-02T16:23:16.594699+0000","last_active":"2024-04-02T16:23:17.605518+0000","last_peered":"2024-04-02T16:23:17.605518+0000","last_clean":"2024-04-02T16:23:17.605518+0000","last_became_active":"2024-04-02T16:23:16.594435+0000","last_became_peered":"2024-04-02T16:23:16.594435+0000","last_unstale":"2024-04-02T16:23:17.605518+0000","last_undegraded":"2024-04-02T16:23:17.605518+0000","last_fullsized":"2024-04-02T16:23:17.605518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:55.457914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.591807+0000","last_change":"2024-04-02T16:23:15.597593+0000","last_active":"2024-04-02T16:23:16.591807+0000","last_peered":"2024-04-02T16:23:16.591807+0000","last_clean":"2024-04-02T16:23:16.591807+0000","last_became_active":"2024-04-02T16:23:15.597295+0000","last_became_peered":"2024-04-02T16:23:15.597295+0000","last_unstale":"2024-04-02T16:23:16.591807+0000","last_undegraded":"2024-04-02T16:23:16.591807+0000","last_fullsized":"2024-04-02T16:23:16.591807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:43.376692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610775+0000","last_change":"2024-04-02T16:23:16.604883+0000","last_active":"2024-04-02T16:23:16.610775+0000","last_peered":"2024-04-02T16:23:16.610775+0000","last_clean":"2024-04-02T16:23:16.610775+0000","last_became_active":"2024-04-02T16:23:16.604570+0000","last_became_peered":"2024-04-02T16:23:16.604570+0000","last_unstale":"2024-04-02T16:23:16.610775+0000","last_undegraded":"2024-04-02T16:23:16.610775+0000","last_fullsized":"2024-04-02T16:23:16.610775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:57:04.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605469+0000","last_change":"2024-04-02T16:23:15.598860+0000","last_active":"2024-04-02T16:23:17.605469+0000","last_peered":"2024-04-02T16:23:17.605469+0000","last_clean":"2024-04-02T16:23:17.605469+0000","last_became_active":"2024-04-02T16:23:15.598543+0000","last_became_peered":"2024-04-02T16:23:15.598543+0000","last_unstale":"2024-04-02T16:23:17.605469+0000","last_undegraded":"2024-04-02T16:23:17.605469+0000","last_fullsized":"2024-04-02T16:23:17.605469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:01:35.512859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.611080+0000","last_change":"2024-04-02T16:23:16.607205+0000","last_active":"2024-04-02T16:23:16.611080+0000","last_peered":"2024-04-02T16:23:16.611080+0000","last_clean":"2024-04-02T16:23:16.611080+0000","last_became_active":"2024-04-02T16:23:16.606406+0000","last_became_peered":"2024-04-02T16:23:16.606406+0000","last_unstale":"2024-04-02T16:23:16.611080+0000","last_undegraded":"2024-04-02T16:23:16.611080+0000","last_fullsized":"2024-04-02T16:23:16.611080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:44.379025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605342+0000","last_change":"2024-04-02T16:23:15.594276+0000","last_active":"2024-04-02T16:23:17.605342+0000","last_peered":"2024-04-02T16:23:17.605342+0000","last_clean":"2024-04-02T16:23:17.605342+0000","last_became_active":"2024-04-02T16:23:15.593980+0000","last_became_peered":"2024-04-02T16:23:15.593980+0000","last_unstale":"2024-04-02T16:23:17.605342+0000","last_undegraded":"2024-04-02T16:23:17.605342+0000","last_fullsized":"2024-04-02T16:23:17.605342+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:41.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587208+0000","last_change":"2024-04-02T16:23:16.598645+0000","last_active":"2024-04-02T16:23:17.587208+0000","last_peered":"2024-04-02T16:23:17.587208+0000","last_clean":"2024-04-02T16:23:17.587208+0000","last_became_active":"2024-04-02T16:23:16.598375+0000","last_became_peered":"2024-04-02T16:23:16.598375+0000","last_unstale":"2024-04-02T16:23:17.587208+0000","last_undegraded":"2024-04-02T16:23:17.587208+0000","last_fullsized":"2024-04-02T16:23:17.587208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:16.140068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.593039+0000","last_change":"2024-04-02T16:23:15.594935+0000","last_active":"2024-04-02T16:23:16.593039+0000","last_peered":"2024-04-02T16:23:16.593039+0000","last_clean":"2024-04-02T16:23:16.593039+0000","last_became_active":"2024-04-02T16:23:15.594618+0000","last_became_peered":"2024-04-02T16:23:15.594618+0000","last_unstale":"2024-04-02T16:23:16.593039+0000","last_undegraded":"2024-04-02T16:23:16.593039+0000","last_fullsized":"2024-04-02T16:23:16.593039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:44.382023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586135+0000","last_change":"2024-04-02T16:23:16.597788+0000","last_active":"2024-04-02T16:23:17.586135+0000","last_peered":"2024-04-02T16:23:17.586135+0000","last_clean":"2024-04-02T16:23:17.586135+0000","last_became_active":"2024-04-02T16:23:16.597423+0000","last_became_peered":"2024-04-02T16:23:16.597423+0000","last_unstale":"2024-04-02T16:23:17.586135+0000","last_undegraded":"2024-04-02T16:23:17.586135+0000","last_fullsized":"2024-04-02T16:23:17.586135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:59:33.161559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605529+0000","last_change":"2024-04-02T16:23:15.600429+0000","last_active":"2024-04-02T16:23:17.605529+0000","last_peered":"2024-04-02T16:23:17.605529+0000","last_clean":"2024-04-02T16:23:17.605529+0000","last_became_active":"2024-04-02T16:23:15.600140+0000","last_became_peered":"2024-04-02T16:23:15.600140+0000","last_unstale":"2024-04-02T16:23:17.605529+0000","last_undegraded":"2024-04-02T16:23:17.605529+0000","last_fullsized":"2024-04-02T16:23:17.605529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:32:19.499561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.606105+0000","last_change":"2024-04-02T16:23:16.601486+0000","last_active":"2024-04-02T16:23:16.606105+0000","last_peered":"2024-04-02T16:23:16.606105+0000","last_clean":"2024-04-02T16:23:16.606105+0000","last_became_active":"2024-04-02T16:23:16.601199+0000","last_became_peered":"2024-04-02T16:23:16.601199+0000","last_unstale":"2024-04-02T16:23:16.606105+0000","last_undegraded":"2024-04-02T16:23:16.606105+0000","last_fullsized":"2024-04-02T16:23:16.606105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:17.861607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587423+0000","last_change":"2024-04-02T16:23:15.588940+0000","last_active":"2024-04-02T16:23:17.587423+0000","last_peered":"2024-04-02T16:23:17.587423+0000","last_clean":"2024-04-02T16:23:17.587423+0000","last_became_active":"2024-04-02T16:23:15.588413+0000","last_became_peered":"2024-04-02T16:23:15.588413+0000","last_unstale":"2024-04-02T16:23:17.587423+0000","last_undegraded":"2024-04-02T16:23:17.587423+0000","last_fullsized":"2024-04-02T16:23:17.587423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:45.252611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.613690+0000","last_change":"2024-04-02T16:23:16.611229+0000","last_active":"2024-04-02T16:23:16.613690+0000","last_peered":"2024-04-02T16:23:16.613690+0000","last_clean":"2024-04-02T16:23:16.613690+0000","last_became_active":"2024-04-02T16:23:16.610937+0000","last_became_peered":"2024-04-02T16:23:16.610937+0000","last_unstale":"2024-04-02T16:23:16.613690+0000","last_undegraded":"2024-04-02T16:23:16.613690+0000","last_fullsized":"2024-04-02T16:23:16.613690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:57.683621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605346+0000","last_change":"2024-04-02T16:23:15.593263+0000","last_active":"2024-04-02T16:23:17.605346+0000","last_peered":"2024-04-02T16:23:17.605346+0000","last_clean":"2024-04-02T16:23:17.605346+0000","last_became_active":"2024-04-02T16:23:15.592965+0000","last_became_peered":"2024-04-02T16:23:15.592965+0000","last_unstale":"2024-04-02T16:23:17.605346+0000","last_undegraded":"2024-04-02T16:23:17.605346+0000","last_fullsized":"2024-04-02T16:23:17.605346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.132590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586282+0000","last_change":"2024-04-02T16:23:16.611900+0000","last_active":"2024-04-02T16:23:17.586282+0000","last_peered":"2024-04-02T16:23:17.586282+0000","last_clean":"2024-04-02T16:23:17.586282+0000","last_became_active":"2024-04-02T16:23:16.606097+0000","last_became_peered":"2024-04-02T16:23:16.606097+0000","last_unstale":"2024-04-02T16:23:17.586282+0000","last_undegraded":"2024-04-02T16:23:17.586282+0000","last_fullsized":"2024-04-02T16:23:17.586282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:17.719148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.596371+0000","last_change":"2024-04-02T16:23:15.598453+0000","last_active":"2024-04-02T16:23:16.596371+0000","last_peered":"2024-04-02T16:23:16.596371+0000","last_clean":"2024-04-02T16:23:16.596371+0000","last_became_active":"2024-04-02T16:23:15.598170+0000","last_became_peered":"2024-04-02T16:23:15.598170+0000","last_unstale":"2024-04-02T16:23:16.596371+0000","last_undegraded":"2024-04-02T16:23:16.596371+0000","last_fullsized":"2024-04-02T16:23:16.596371+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:54.455581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.614246+0000","last_change":"2024-04-02T16:23:16.607993+0000","last_active":"2024-04-02T16:23:16.614246+0000","last_peered":"2024-04-02T16:23:16.614246+0000","last_clean":"2024-04-02T16:23:16.614246+0000","last_became_active":"2024-04-02T16:23:16.607667+0000","last_became_peered":"2024-04-02T16:23:16.607667+0000","last_unstale":"2024-04-02T16:23:16.614246+0000","last_undegraded":"2024-04-02T16:23:16.614246+0000","last_fullsized":"2024-04-02T16:23:16.614246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:46.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.596713+0000","last_change":"2024-04-02T16:23:15.600399+0000","last_active":"2024-04-02T16:23:16.596713+0000","last_peered":"2024-04-02T16:23:16.596713+0000","last_clean":"2024-04-02T16:23:16.596713+0000","last_became_active":"2024-04-02T16:23:15.599981+0000","last_became_peered":"2024-04-02T16:23:15.599981+0000","last_unstale":"2024-04-02T16:23:16.596713+0000","last_undegraded":"2024-04-02T16:23:16.596713+0000","last_fullsized":"2024-04-02T16:23:16.596713+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:44.382023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.602508+0000","last_change":"2024-04-02T16:23:16.598611+0000","last_active":"2024-04-02T16:23:16.602508+0000","last_peered":"2024-04-02T16:23:16.602508+0000","last_clean":"2024-04-02T16:23:16.602508+0000","last_became_active":"2024-04-02T16:23:16.598335+0000","last_became_peered":"2024-04-02T16:23:16.598335+0000","last_unstale":"2024-04-02T16:23:16.602508+0000","last_undegraded":"2024-04-02T16:23:16.602508+0000","last_fullsized":"2024-04-02T16:23:16.602508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:46.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.592877+0000","last_change":"2024-04-02T16:23:15.600550+0000","last_active":"2024-04-02T16:23:16.592877+0000","last_peered":"2024-04-02T16:23:16.592877+0000","last_clean":"2024-04-02T16:23:16.592877+0000","last_became_active":"2024-04-02T16:23:15.600266+0000","last_became_peered":"2024-04-02T16:23:15.600266+0000","last_unstale":"2024-04-02T16:23:16.592877+0000","last_undegraded":"2024-04-02T16:23:16.592877+0000","last_fullsized":"2024-04-02T16:23:16.592877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:54.455581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587618+0000","last_change":"2024-04-02T16:23:16.600255+0000","last_active":"2024-04-02T16:23:17.587618+0000","last_peered":"2024-04-02T16:23:17.587618+0000","last_clean":"2024-04-02T16:23:17.587618+0000","last_became_active":"2024-04-02T16:23:16.599861+0000","last_became_peered":"2024-04-02T16:23:16.599861+0000","last_unstale":"2024-04-02T16:23:17.587618+0000","last_undegraded":"2024-04-02T16:23:17.587618+0000","last_fullsized":"2024-04-02T16:23:17.587618+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:32.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587536+0000","last_change":"2024-04-02T16:23:15.597125+0000","last_active":"2024-04-02T16:23:17.587536+0000","last_peered":"2024-04-02T16:23:17.587536+0000","last_clean":"2024-04-02T16:23:17.587536+0000","last_became_active":"2024-04-02T16:23:15.596724+0000","last_became_peered":"2024-04-02T16:23:15.596724+0000","last_unstale":"2024-04-02T16:23:17.587536+0000","last_undegraded":"2024-04-02T16:23:17.587536+0000","last_fullsized":"2024-04-02T16:23:17.587536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:43.376692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.603506+0000","last_change":"2024-04-02T16:23:16.598737+0000","last_active":"2024-04-02T16:23:16.603506+0000","last_peered":"2024-04-02T16:23:16.603506+0000","last_clean":"2024-04-02T16:23:16.603506+0000","last_became_active":"2024-04-02T16:23:16.598382+0000","last_became_peered":"2024-04-02T16:23:16.598382+0000","last_unstale":"2024-04-02T16:23:16.603506+0000","last_undegraded":"2024-04-02T16:23:16.603506+0000","last_fullsized":"2024-04-02T16:23:16.603506+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:44.379025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585724+0000","last_change":"2024-04-02T16:23:15.592737+0000","last_active":"2024-04-02T16:23:17.585724+0000","last_peered":"2024-04-02T16:23:17.585724+0000","last_clean":"2024-04-02T16:23:17.585724+0000","last_became_active":"2024-04-02T16:23:15.592438+0000","last_became_peered":"2024-04-02T16:23:15.592438+0000","last_unstale":"2024-04-02T16:23:17.585724+0000","last_undegraded":"2024-04-02T16:23:17.585724+0000","last_fullsized":"2024-04-02T16:23:17.585724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:56.681288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587359+0000","last_change":"2024-04-02T16:23:16.606633+0000","last_active":"2024-04-02T16:23:17.587359+0000","last_peered":"2024-04-02T16:23:17.587359+0000","last_clean":"2024-04-02T16:23:17.587359+0000","last_became_active":"2024-04-02T16:23:16.606368+0000","last_became_peered":"2024-04-02T16:23:16.606368+0000","last_unstale":"2024-04-02T16:23:17.587359+0000","last_undegraded":"2024-04-02T16:23:17.587359+0000","last_fullsized":"2024-04-02T16:23:17.587359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:59:33.161559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.593510+0000","last_change":"2024-04-02T16:23:15.597318+0000","last_active":"2024-04-02T16:23:16.593510+0000","last_peered":"2024-04-02T16:23:16.593510+0000","last_clean":"2024-04-02T16:23:16.593510+0000","last_became_active":"2024-04-02T16:23:15.597016+0000","last_became_peered":"2024-04-02T16:23:15.597016+0000","last_unstale":"2024-04-02T16:23:16.593510+0000","last_undegraded":"2024-04-02T16:23:16.593510+0000","last_fullsized":"2024-04-02T16:23:16.593510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:51.685925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.593131+0000","last_change":"2024-04-02T16:21:49.851609+0000","last_active":"2024-04-02T16:23:16.593131+0000","last_peered":"2024-04-02T16:23:16.593131+0000","last_clean":"2024-04-02T16:23:16.593131+0000","last_became_active":"2024-04-02T16:21:49.850564+0000","last_became_peered":"2024-04-02T16:21:49.850564+0000","last_unstale":"2024-04-02T16:23:16.593131+0000","last_undegraded":"2024-04-02T16:23:16.593131+0000","last_fullsized":"2024-04-02T16:23:16.593131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_clean_scrub_stamp":"2024-04-02T16:21:48.692321+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.825754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":191225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":198981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":781045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:23:19.613 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:23:19.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.750+0000 7ff9b48e4640 1 Processor -- start 2024-04-02T16:23:19.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.750+0000 7ff9b48e4640 1 -- start start 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9b48e4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9b00b22d0 0x7ff9b015b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9b48e4640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b015fb10 con 0x7ff9b013dfb0 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9b48e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b013e390 con 0x7ff9b015bd70 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9b48e4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b015ed50 con 0x7ff9b00b22d0 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9ae59c640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7ff9b013dfb0 0x7ff9b00b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42478/0 (socket says 172.21.15.50:42478) 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9ae59c640 1 -- 172.21.15.50:0/1872293503 learned_addr learned my addr 172.21.15.50:0/1872293503 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9add9b640 1 --2- 172.21.15.50:0/1872293503 >> v2:172.21.15.188:3300/0 conn(0x7ff9b00b22d0 0x7ff9b015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615061324 0 0) 0x7ff9b015fb10 con 0x7ff9b013dfb0 2024-04-02T16:23:19.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.751+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff99c003520 con 0x7ff9b013dfb0 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1568049739 0 0) 0x7ff99c003520 con 0x7ff9b013dfb0 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 >> v2:172.21.15.188:3300/0 conn(0x7ff9b00b22d0 msgr2=0x7ff9b015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 --2- 172.21.15.50:0/1872293503 >> v2:172.21.15.188:3300/0 conn(0x7ff9b00b22d0 0x7ff9b015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 >> v1:172.21.15.188:6789/0 conn(0x7ff9b015bd70 legacy=0x7ff9b00ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b015f030 con 0x7ff9b013dfb0 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7ff9a00019f0 con 0x7ff9b013dfb0 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 >> v1:172.21.15.50:6789/0 conn(0x7ff9b013dfb0 legacy=0x7ff9b00b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c0039b0 0x7ff99c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff99c0043b0 0x7ff99c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c005120 0x7ff99c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b015ed50 con 0x7ff99c0043b0 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c00c060 con 0x7ff99c005120 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c00c260 con 0x7ff99c0039b0 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9add9b640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff99c0043b0 0x7ff99c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9aed9d640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c005120 0x7ff99c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ae59c640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c0039b0 0x7ff99c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9add9b640 1 -- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c0039b0 msgr2=0x7ff99c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9add9b640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c0039b0 0x7ff99c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9add9b640 1 -- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c005120 msgr2=0x7ff99c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9add9b640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c005120 0x7ff99c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9add9b640 1 -- 172.21.15.50:0/1872293503 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c003520 con 0x7ff99c0043b0 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9aed9d640 1 -- 172.21.15.50:0/1872293503 reap_dead start 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.752+0000 7ff9ae59c640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c0039b0 0x7ff99c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.753+0000 7ff9add9b640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff99c0043b0 0x7ff99c004880 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e6dd89761d2748 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.753+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998039070 con 0x7ff99c0043b0 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.753+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99802c6b0 con 0x7ff99c0043b0 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.753+0000 7ff9ad59a640 1 -- 172.21.15.50:0/1872293503 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998009690 con 0x7ff99c0043b0 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.754+0000 7ff9b48e4640 1 -- 172.21.15.50:0/1872293503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff99c0043b0 msgr2=0x7ff99c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.754+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff99c0043b0 0x7ff99c004880 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff99c005830 comp rx=0 tx=0).stop 2024-04-02T16:23:19.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.754+0000 7ff9b48e4640 1 -- 172.21.15.50:0/1872293503 shutdown_connections 2024-04-02T16:23:19.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.754+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/1872293503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff99c0043b0 0x7ff99c004880 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.754+0000 7ff9b48e4640 1 -- 172.21.15.50:0/1872293503 >> 172.21.15.50:0/1872293503 conn(0x7ff9b006ebf0 msgr2=0x7ff9b006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:19.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.754+0000 7ff9b48e4640 1 -- 172.21.15.50:0/1872293503 shutdown_connections 2024-04-02T16:23:19.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.754+0000 7ff9b48e4640 1 -- 172.21.15.50:0/1872293503 wait complete. 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.755+0000 7ff9b48e4640 1 Processor -- start 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.755+0000 7ff9b48e4640 1 -- start start 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9b48e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0043b0 0x7ff9b01f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9b48e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b0144130 0x7ff9b0140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9b48e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b015a3a0 0x7ff9b015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9b48e4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b015c1a0 con 0x7ff9b0144130 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9b48e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b015f5f0 con 0x7ff99c0043b0 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9b48e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b015c320 con 0x7ff9b015a3a0 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b0144130 0x7ff9b0140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b0144130 0x7ff9b0140800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41010/0 (socket says 172.21.15.50:41010) 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 -- 172.21.15.50:0/2634005527 learned_addr learned my addr 172.21.15.50:0/2634005527 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9aed9d640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b015a3a0 0x7ff9b015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9ae59c640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0043b0 0x7ff9b01f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 -- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b015a3a0 msgr2=0x7ff9b015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b015a3a0 0x7ff9b015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 -- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0043b0 msgr2=0x7ff9b01f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0043b0 0x7ff9b01f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 -- 172.21.15.50:0/2634005527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b01411a0 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9ae59c640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0043b0 0x7ff9b01f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.756+0000 7ff9add9b640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b0144130 0x7ff9b0140800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff998004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.757+0000 7ff9a6ffd640 1 -- 172.21.15.50:0/2634005527 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998039050 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.757+0000 7ff9a6ffd640 1 -- 172.21.15.50:0/2634005527 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff998045070 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.757+0000 7ff9a6ffd640 1 -- 172.21.15.50:0/2634005527 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980044e0 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.757+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b0141960 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.757+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b0140d40 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.758+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b00756f0 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.758+0000 7ff9a6ffd640 1 -- 172.21.15.50:0/2634005527 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff998003a50 con 0x7ff9b0144130 2024-04-02T16:23:19.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.758+0000 7ff9a6ffd640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff99006c8f0 0x7ff99006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:19.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.759+0000 7ff9a6ffd640 1 -- 172.21.15.50:0/2634005527 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff9980adb70 con 0x7ff9b0144130 2024-04-02T16:23:19.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.759+0000 7ff9ae59c640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff99006c8f0 0x7ff99006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:19.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.759+0000 7ff9ae59c640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff99006c8f0 0x7ff99006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0002230 tx=0x7ff9a0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:19.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.763+0000 7ff9a6ffd640 1 -- 172.21.15.50:0/2634005527 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff99804f020 con 0x7ff9b0144130 2024-04-02T16:23:19.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.944+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 --> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9b0141480 con 0x7ff99006c8f0 2024-04-02T16:23:19.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.951+0000 7ff9a6ffd640 1 -- 172.21.15.50:0/2634005527 <== mgr.4112 v2:172.21.15.188:6832/2785600878 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7ff9b0141480 con 0x7ff99006c8f0 2024-04-02T16:23:19.953 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:19.954 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-04-02T16:23:19.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff99006c8f0 msgr2=0x7ff99006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff99006c8f0 0x7ff99006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0002230 tx=0x7ff9a0009040 comp rx=0 tx=0).stop 2024-04-02T16:23:19.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b0144130 msgr2=0x7ff9b0140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:19.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b0144130 0x7ff9b0140800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff998004730 comp rx=0 tx=0).stop 2024-04-02T16:23:19.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 shutdown_connections 2024-04-02T16:23:19.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9b015a3a0 0x7ff9b015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7ff99006c8f0 0x7ff99006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9b0144130 0x7ff9b0140800 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 --2- 172.21.15.50:0/2634005527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0043b0 0x7ff9b01f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:19.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 >> 172.21.15.50:0/2634005527 conn(0x7ff9b006ebf0 msgr2=0x7ff9b00ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:19.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 shutdown_connections 2024-04-02T16:23:19.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:19.954+0000 7ff9b48e4640 1 -- 172.21.15.50:0/2634005527 wait complete. 2024-04-02T16:23:19.964 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-02T16:23:18.684617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237472,"kb_used_data":4660,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737248,"statfs":{"total":773094113280,"available":772850941952,"internally_reserved":0,"allocated":4771840,"data_stored":2717392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.110661"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586188+0000","last_change":"2024-04-02T16:23:16.610917+0000","last_active":"2024-04-02T16:23:17.586188+0000","last_peered":"2024-04-02T16:23:17.586188+0000","last_clean":"2024-04-02T16:23:17.586188+0000","last_became_active":"2024-04-02T16:23:16.610588+0000","last_became_peered":"2024-04-02T16:23:16.610588+0000","last_unstale":"2024-04-02T16:23:17.586188+0000","last_undegraded":"2024-04-02T16:23:17.586188+0000","last_fullsized":"2024-04-02T16:23:17.586188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:30:47.908172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586255+0000","last_change":"2024-04-02T16:23:16.610552+0000","last_active":"2024-04-02T16:23:17.586255+0000","last_peered":"2024-04-02T16:23:17.586255+0000","last_clean":"2024-04-02T16:23:17.586255+0000","last_became_active":"2024-04-02T16:23:16.604220+0000","last_became_peered":"2024-04-02T16:23:16.604220+0000","last_unstale":"2024-04-02T16:23:17.586255+0000","last_undegraded":"2024-04-02T16:23:17.586255+0000","last_fullsized":"2024-04-02T16:23:17.586255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:36:28.419284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.611086+0000","last_change":"2024-04-02T16:23:16.606945+0000","last_active":"2024-04-02T16:23:16.611086+0000","last_peered":"2024-04-02T16:23:16.611086+0000","last_clean":"2024-04-02T16:23:16.611086+0000","last_became_active":"2024-04-02T16:23:16.606617+0000","last_became_peered":"2024-04-02T16:23:16.606617+0000","last_unstale":"2024-04-02T16:23:16.611086+0000","last_undegraded":"2024-04-02T16:23:16.611086+0000","last_fullsized":"2024-04-02T16:23:16.611086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:23.481064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605861+0000","last_change":"2024-04-02T16:23:16.603972+0000","last_active":"2024-04-02T16:23:17.605861+0000","last_peered":"2024-04-02T16:23:17.605861+0000","last_clean":"2024-04-02T16:23:17.605861+0000","last_became_active":"2024-04-02T16:23:16.602669+0000","last_became_peered":"2024-04-02T16:23:16.602669+0000","last_unstale":"2024-04-02T16:23:17.605861+0000","last_undegraded":"2024-04-02T16:23:17.605861+0000","last_fullsized":"2024-04-02T16:23:17.605861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:10.831124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605298+0000","last_change":"2024-04-02T16:23:16.591590+0000","last_active":"2024-04-02T16:23:17.605298+0000","last_peered":"2024-04-02T16:23:17.605298+0000","last_clean":"2024-04-02T16:23:17.605298+0000","last_became_active":"2024-04-02T16:23:16.591314+0000","last_became_peered":"2024-04-02T16:23:16.591314+0000","last_unstale":"2024-04-02T16:23:17.605298+0000","last_undegraded":"2024-04-02T16:23:17.605298+0000","last_fullsized":"2024-04-02T16:23:17.605298+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:59.183875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.600476+0000","last_change":"2024-04-02T16:23:16.596994+0000","last_active":"2024-04-02T16:23:16.600476+0000","last_peered":"2024-04-02T16:23:16.600476+0000","last_clean":"2024-04-02T16:23:16.600476+0000","last_became_active":"2024-04-02T16:23:16.596484+0000","last_became_peered":"2024-04-02T16:23:16.596484+0000","last_unstale":"2024-04-02T16:23:16.600476+0000","last_undegraded":"2024-04-02T16:23:16.600476+0000","last_fullsized":"2024-04-02T16:23:16.600476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:59.966073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586860+0000","last_change":"2024-04-02T16:23:16.598667+0000","last_active":"2024-04-02T16:23:17.586860+0000","last_peered":"2024-04-02T16:23:17.586860+0000","last_clean":"2024-04-02T16:23:17.586860+0000","last_became_active":"2024-04-02T16:23:16.598394+0000","last_became_peered":"2024-04-02T16:23:16.598394+0000","last_unstale":"2024-04-02T16:23:17.586860+0000","last_undegraded":"2024-04-02T16:23:17.586860+0000","last_fullsized":"2024-04-02T16:23:17.586860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:50.721161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585736+0000","last_change":"2024-04-02T16:23:16.610689+0000","last_active":"2024-04-02T16:23:17.585736+0000","last_peered":"2024-04-02T16:23:17.585736+0000","last_clean":"2024-04-02T16:23:17.585736+0000","last_became_active":"2024-04-02T16:23:16.605655+0000","last_became_peered":"2024-04-02T16:23:16.605655+0000","last_unstale":"2024-04-02T16:23:17.585736+0000","last_undegraded":"2024-04-02T16:23:17.585736+0000","last_fullsized":"2024-04-02T16:23:17.585736+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:17:49.482366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610813+0000","last_change":"2024-04-02T16:23:16.606933+0000","last_active":"2024-04-02T16:23:16.610813+0000","last_peered":"2024-04-02T16:23:16.610813+0000","last_clean":"2024-04-02T16:23:16.610813+0000","last_became_active":"2024-04-02T16:23:16.606627+0000","last_became_peered":"2024-04-02T16:23:16.606627+0000","last_unstale":"2024-04-02T16:23:16.610813+0000","last_undegraded":"2024-04-02T16:23:16.610813+0000","last_fullsized":"2024-04-02T16:23:16.610813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:52.688258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587186+0000","last_change":"2024-04-02T16:23:16.607054+0000","last_active":"2024-04-02T16:23:17.587186+0000","last_peered":"2024-04-02T16:23:17.587186+0000","last_clean":"2024-04-02T16:23:17.587186+0000","last_became_active":"2024-04-02T16:23:16.606207+0000","last_became_peered":"2024-04-02T16:23:16.606207+0000","last_unstale":"2024-04-02T16:23:17.587186+0000","last_undegraded":"2024-04-02T16:23:17.587186+0000","last_fullsized":"2024-04-02T16:23:17.587186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:33:49.551083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586799+0000","last_change":"2024-04-02T16:23:16.603903+0000","last_active":"2024-04-02T16:23:17.586799+0000","last_peered":"2024-04-02T16:23:17.586799+0000","last_clean":"2024-04-02T16:23:17.586799+0000","last_became_active":"2024-04-02T16:23:16.603621+0000","last_became_peered":"2024-04-02T16:23:16.603621+0000","last_unstale":"2024-04-02T16:23:17.586799+0000","last_undegraded":"2024-04-02T16:23:17.586799+0000","last_fullsized":"2024-04-02T16:23:17.586799+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:59.438279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610804+0000","last_change":"2024-04-02T16:23:16.607184+0000","last_active":"2024-04-02T16:23:16.610804+0000","last_peered":"2024-04-02T16:23:16.610804+0000","last_clean":"2024-04-02T16:23:16.610804+0000","last_became_active":"2024-04-02T16:23:16.606669+0000","last_became_peered":"2024-04-02T16:23:16.606669+0000","last_unstale":"2024-04-02T16:23:16.610804+0000","last_undegraded":"2024-04-02T16:23:16.610804+0000","last_fullsized":"2024-04-02T16:23:16.610804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:13.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586801+0000","last_change":"2024-04-02T16:23:16.605049+0000","last_active":"2024-04-02T16:23:17.586801+0000","last_peered":"2024-04-02T16:23:17.586801+0000","last_clean":"2024-04-02T16:23:17.586801+0000","last_became_active":"2024-04-02T16:23:16.604743+0000","last_became_peered":"2024-04-02T16:23:16.604743+0000","last_unstale":"2024-04-02T16:23:17.586801+0000","last_undegraded":"2024-04-02T16:23:17.586801+0000","last_fullsized":"2024-04-02T16:23:17.586801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:24.435217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587474+0000","last_change":"2024-04-02T16:23:16.600887+0000","last_active":"2024-04-02T16:23:17.587474+0000","last_peered":"2024-04-02T16:23:17.587474+0000","last_clean":"2024-04-02T16:23:17.587474+0000","last_became_active":"2024-04-02T16:23:16.600585+0000","last_became_peered":"2024-04-02T16:23:16.600585+0000","last_unstale":"2024-04-02T16:23:17.587474+0000","last_undegraded":"2024-04-02T16:23:17.587474+0000","last_fullsized":"2024-04-02T16:23:17.587474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:38.558835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587238+0000","last_change":"2024-04-02T16:23:16.598969+0000","last_active":"2024-04-02T16:23:17.587238+0000","last_peered":"2024-04-02T16:23:17.587238+0000","last_clean":"2024-04-02T16:23:17.587238+0000","last_became_active":"2024-04-02T16:23:16.597796+0000","last_became_peered":"2024-04-02T16:23:16.597796+0000","last_unstale":"2024-04-02T16:23:17.587238+0000","last_undegraded":"2024-04-02T16:23:17.587238+0000","last_fullsized":"2024-04-02T16:23:17.587238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:00.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.607772+0000","last_change":"2024-04-02T16:23:16.601475+0000","last_active":"2024-04-02T16:23:16.607772+0000","last_peered":"2024-04-02T16:23:16.607772+0000","last_clean":"2024-04-02T16:23:16.607772+0000","last_became_active":"2024-04-02T16:23:16.601181+0000","last_became_peered":"2024-04-02T16:23:16.601181+0000","last_unstale":"2024-04-02T16:23:16.607772+0000","last_undegraded":"2024-04-02T16:23:16.607772+0000","last_fullsized":"2024-04-02T16:23:16.607772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:23.481064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.607780+0000","last_change":"2024-04-02T16:23:16.602426+0000","last_active":"2024-04-02T16:23:16.607780+0000","last_peered":"2024-04-02T16:23:16.607780+0000","last_clean":"2024-04-02T16:23:16.607780+0000","last_became_active":"2024-04-02T16:23:16.602147+0000","last_became_peered":"2024-04-02T16:23:16.602147+0000","last_unstale":"2024-04-02T16:23:16.607780+0000","last_undegraded":"2024-04-02T16:23:16.607780+0000","last_fullsized":"2024-04-02T16:23:16.607780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:46.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.613581+0000","last_change":"2024-04-02T16:23:16.600782+0000","last_active":"2024-04-02T16:23:16.613581+0000","last_peered":"2024-04-02T16:23:16.613581+0000","last_clean":"2024-04-02T16:23:16.613581+0000","last_became_active":"2024-04-02T16:23:16.600521+0000","last_became_peered":"2024-04-02T16:23:16.600521+0000","last_unstale":"2024-04-02T16:23:16.613581+0000","last_undegraded":"2024-04-02T16:23:16.613581+0000","last_fullsized":"2024-04-02T16:23:16.613581+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:13.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605379+0000","last_change":"2024-04-02T16:23:16.604296+0000","last_active":"2024-04-02T16:23:17.605379+0000","last_peered":"2024-04-02T16:23:17.605379+0000","last_clean":"2024-04-02T16:23:17.605379+0000","last_became_active":"2024-04-02T16:23:16.603883+0000","last_became_peered":"2024-04-02T16:23:16.603883+0000","last_unstale":"2024-04-02T16:23:17.605379+0000","last_undegraded":"2024-04-02T16:23:17.605379+0000","last_fullsized":"2024-04-02T16:23:17.605379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:47.341504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.596264+0000","last_change":"2024-04-02T16:23:16.590853+0000","last_active":"2024-04-02T16:23:16.596264+0000","last_peered":"2024-04-02T16:23:16.596264+0000","last_clean":"2024-04-02T16:23:16.596264+0000","last_became_active":"2024-04-02T16:23:16.590574+0000","last_became_peered":"2024-04-02T16:23:16.590574+0000","last_unstale":"2024-04-02T16:23:16.596264+0000","last_undegraded":"2024-04-02T16:23:16.596264+0000","last_fullsized":"2024-04-02T16:23:16.596264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:54:47.341504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586985+0000","last_change":"2024-04-02T16:23:16.599240+0000","last_active":"2024-04-02T16:23:17.586985+0000","last_peered":"2024-04-02T16:23:17.586985+0000","last_clean":"2024-04-02T16:23:17.586985+0000","last_became_active":"2024-04-02T16:23:16.598954+0000","last_became_peered":"2024-04-02T16:23:16.598954+0000","last_unstale":"2024-04-02T16:23:17.586985+0000","last_undegraded":"2024-04-02T16:23:17.586985+0000","last_fullsized":"2024-04-02T16:23:17.586985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:17:49.482366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587828+0000","last_change":"2024-04-02T16:23:16.594822+0000","last_active":"2024-04-02T16:23:17.587828+0000","last_peered":"2024-04-02T16:23:17.587828+0000","last_clean":"2024-04-02T16:23:17.587828+0000","last_became_active":"2024-04-02T16:23:16.594534+0000","last_became_peered":"2024-04-02T16:23:16.594534+0000","last_unstale":"2024-04-02T16:23:17.587828+0000","last_undegraded":"2024-04-02T16:23:17.587828+0000","last_fullsized":"2024-04-02T16:23:17.587828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:24.435217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610368+0000","last_change":"2024-04-02T16:23:16.604204+0000","last_active":"2024-04-02T16:23:16.610368+0000","last_peered":"2024-04-02T16:23:16.610368+0000","last_clean":"2024-04-02T16:23:16.610368+0000","last_became_active":"2024-04-02T16:23:16.603946+0000","last_became_peered":"2024-04-02T16:23:16.603946+0000","last_unstale":"2024-04-02T16:23:16.610368+0000","last_undegraded":"2024-04-02T16:23:16.610368+0000","last_fullsized":"2024-04-02T16:23:16.610368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:17.861607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605278+0000","last_change":"2024-04-02T16:23:16.603268+0000","last_active":"2024-04-02T16:23:17.605278+0000","last_peered":"2024-04-02T16:23:17.605278+0000","last_clean":"2024-04-02T16:23:17.605278+0000","last_became_active":"2024-04-02T16:23:16.602949+0000","last_became_peered":"2024-04-02T16:23:16.602949+0000","last_unstale":"2024-04-02T16:23:17.605278+0000","last_undegraded":"2024-04-02T16:23:17.605278+0000","last_fullsized":"2024-04-02T16:23:17.605278+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:24.522597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.615529+0000","last_change":"2024-04-02T16:23:16.611562+0000","last_active":"2024-04-02T16:23:16.615529+0000","last_peered":"2024-04-02T16:23:16.615529+0000","last_clean":"2024-04-02T16:23:16.615529+0000","last_became_active":"2024-04-02T16:23:16.611263+0000","last_became_peered":"2024-04-02T16:23:16.611263+0000","last_unstale":"2024-04-02T16:23:16.615529+0000","last_undegraded":"2024-04-02T16:23:16.615529+0000","last_fullsized":"2024-04-02T16:23:16.615529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:44.379025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610323+0000","last_change":"2024-04-02T16:23:16.606339+0000","last_active":"2024-04-02T16:23:16.610323+0000","last_peered":"2024-04-02T16:23:16.610323+0000","last_clean":"2024-04-02T16:23:16.610323+0000","last_became_active":"2024-04-02T16:23:16.604419+0000","last_became_peered":"2024-04-02T16:23:16.604419+0000","last_unstale":"2024-04-02T16:23:16.610323+0000","last_undegraded":"2024-04-02T16:23:16.610323+0000","last_fullsized":"2024-04-02T16:23:16.610323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:57.683621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.597583+0000","last_change":"2024-04-02T16:23:16.593111+0000","last_active":"2024-04-02T16:23:16.597583+0000","last_peered":"2024-04-02T16:23:16.597583+0000","last_clean":"2024-04-02T16:23:16.597583+0000","last_became_active":"2024-04-02T16:23:16.592827+0000","last_became_peered":"2024-04-02T16:23:16.592827+0000","last_unstale":"2024-04-02T16:23:16.597583+0000","last_undegraded":"2024-04-02T16:23:16.597583+0000","last_fullsized":"2024-04-02T16:23:16.597583+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:10.831124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587582+0000","last_change":"2024-04-02T16:23:16.605966+0000","last_active":"2024-04-02T16:23:17.587582+0000","last_peered":"2024-04-02T16:23:17.587582+0000","last_clean":"2024-04-02T16:23:17.587582+0000","last_became_active":"2024-04-02T16:23:16.605495+0000","last_became_peered":"2024-04-02T16:23:16.605495+0000","last_unstale":"2024-04-02T16:23:17.587582+0000","last_undegraded":"2024-04-02T16:23:17.587582+0000","last_fullsized":"2024-04-02T16:23:17.587582+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:30:47.908172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610366+0000","last_change":"2024-04-02T16:23:16.597164+0000","last_active":"2024-04-02T16:23:16.610366+0000","last_peered":"2024-04-02T16:23:16.610366+0000","last_clean":"2024-04-02T16:23:16.610366+0000","last_became_active":"2024-04-02T16:23:16.596897+0000","last_became_peered":"2024-04-02T16:23:16.596897+0000","last_unstale":"2024-04-02T16:23:16.610366+0000","last_undegraded":"2024-04-02T16:23:16.610366+0000","last_fullsized":"2024-04-02T16:23:16.610366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:52.688258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610141+0000","last_change":"2024-04-02T16:23:16.596256+0000","last_active":"2024-04-02T16:23:16.610141+0000","last_peered":"2024-04-02T16:23:16.610141+0000","last_clean":"2024-04-02T16:23:16.610141+0000","last_became_active":"2024-04-02T16:23:16.595967+0000","last_became_peered":"2024-04-02T16:23:16.595967+0000","last_unstale":"2024-04-02T16:23:16.610141+0000","last_undegraded":"2024-04-02T16:23:16.610141+0000","last_fullsized":"2024-04-02T16:23:16.610141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:45.384355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610738+0000","last_change":"2024-04-02T16:23:16.604797+0000","last_active":"2024-04-02T16:23:16.610738+0000","last_peered":"2024-04-02T16:23:16.610738+0000","last_clean":"2024-04-02T16:23:16.610738+0000","last_became_active":"2024-04-02T16:23:16.604507+0000","last_became_peered":"2024-04-02T16:23:16.604507+0000","last_unstale":"2024-04-02T16:23:16.610738+0000","last_undegraded":"2024-04-02T16:23:16.610738+0000","last_fullsized":"2024-04-02T16:23:16.610738+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:26.024041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610898+0000","last_change":"2024-04-02T16:23:16.605620+0000","last_active":"2024-04-02T16:23:16.610898+0000","last_peered":"2024-04-02T16:23:16.610898+0000","last_clean":"2024-04-02T16:23:16.610898+0000","last_became_active":"2024-04-02T16:23:16.605316+0000","last_became_peered":"2024-04-02T16:23:16.605316+0000","last_unstale":"2024-04-02T16:23:16.610898+0000","last_undegraded":"2024-04-02T16:23:16.610898+0000","last_fullsized":"2024-04-02T16:23:16.610898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:26.024041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.607746+0000","last_change":"2024-04-02T16:23:16.601983+0000","last_active":"2024-04-02T16:23:16.607746+0000","last_peered":"2024-04-02T16:23:16.607746+0000","last_clean":"2024-04-02T16:23:16.607746+0000","last_became_active":"2024-04-02T16:23:16.601735+0000","last_became_peered":"2024-04-02T16:23:16.601735+0000","last_unstale":"2024-04-02T16:23:16.607746+0000","last_undegraded":"2024-04-02T16:23:16.607746+0000","last_fullsized":"2024-04-02T16:23:16.607746+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:59.966073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.618649+0000","last_change":"2024-04-02T16:23:15.599944+0000","last_active":"2024-04-02T16:23:16.618649+0000","last_peered":"2024-04-02T16:23:16.618649+0000","last_clean":"2024-04-02T16:23:16.618649+0000","last_became_active":"2024-04-02T16:23:15.599613+0000","last_became_peered":"2024-04-02T16:23:15.599613+0000","last_unstale":"2024-04-02T16:23:16.618649+0000","last_undegraded":"2024-04-02T16:23:16.618649+0000","last_fullsized":"2024-04-02T16:23:16.618649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:23.520265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.612177+0000","last_change":"2024-04-02T16:23:16.606292+0000","last_active":"2024-04-02T16:23:16.612177+0000","last_peered":"2024-04-02T16:23:16.612177+0000","last_clean":"2024-04-02T16:23:16.612177+0000","last_became_active":"2024-04-02T16:23:16.604827+0000","last_became_peered":"2024-04-02T16:23:16.604827+0000","last_unstale":"2024-04-02T16:23:16.612177+0000","last_undegraded":"2024-04-02T16:23:16.612177+0000","last_fullsized":"2024-04-02T16:23:16.612177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:57:04.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.608260+0000","last_change":"2024-04-02T16:23:15.599256+0000","last_active":"2024-04-02T16:23:16.608260+0000","last_peered":"2024-04-02T16:23:16.608260+0000","last_clean":"2024-04-02T16:23:16.608260+0000","last_became_active":"2024-04-02T16:23:15.598974+0000","last_became_peered":"2024-04-02T16:23:15.598974+0000","last_unstale":"2024-04-02T16:23:16.608260+0000","last_undegraded":"2024-04-02T16:23:16.608260+0000","last_fullsized":"2024-04-02T16:23:16.608260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:09.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587777+0000","last_change":"2024-04-02T16:23:16.600799+0000","last_active":"2024-04-02T16:23:17.587777+0000","last_peered":"2024-04-02T16:23:17.587777+0000","last_clean":"2024-04-02T16:23:17.587777+0000","last_became_active":"2024-04-02T16:23:16.600154+0000","last_became_peered":"2024-04-02T16:23:16.600154+0000","last_unstale":"2024-04-02T16:23:17.587777+0000","last_undegraded":"2024-04-02T16:23:17.587777+0000","last_fullsized":"2024-04-02T16:23:17.587777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:39.120379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.618456+0000","last_change":"2024-04-02T16:23:15.598575+0000","last_active":"2024-04-02T16:23:16.618456+0000","last_peered":"2024-04-02T16:23:16.618456+0000","last_clean":"2024-04-02T16:23:16.618456+0000","last_became_active":"2024-04-02T16:23:15.598268+0000","last_became_peered":"2024-04-02T16:23:15.598268+0000","last_unstale":"2024-04-02T16:23:16.618456+0000","last_undegraded":"2024-04-02T16:23:16.618456+0000","last_fullsized":"2024-04-02T16:23:16.618456+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:22.478731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586245+0000","last_change":"2024-04-02T16:23:16.598704+0000","last_active":"2024-04-02T16:23:17.586245+0000","last_peered":"2024-04-02T16:23:17.586245+0000","last_clean":"2024-04-02T16:23:17.586245+0000","last_became_active":"2024-04-02T16:23:16.598449+0000","last_became_peered":"2024-04-02T16:23:16.598449+0000","last_unstale":"2024-04-02T16:23:17.586245+0000","last_undegraded":"2024-04-02T16:23:17.586245+0000","last_fullsized":"2024-04-02T16:23:17.586245+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:53:40.190923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.588148+0000","last_change":"2024-04-02T16:23:15.596983+0000","last_active":"2024-04-02T16:23:17.588148+0000","last_peered":"2024-04-02T16:23:17.588148+0000","last_clean":"2024-04-02T16:23:17.588148+0000","last_became_active":"2024-04-02T16:23:15.596705+0000","last_became_peered":"2024-04-02T16:23:15.596705+0000","last_unstale":"2024-04-02T16:23:17.588148+0000","last_undegraded":"2024-04-02T16:23:17.588148+0000","last_fullsized":"2024-04-02T16:23:17.588148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:12.035337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.614269+0000","last_change":"2024-04-02T16:23:16.608230+0000","last_active":"2024-04-02T16:23:16.614269+0000","last_peered":"2024-04-02T16:23:16.614269+0000","last_clean":"2024-04-02T16:23:16.614269+0000","last_became_active":"2024-04-02T16:23:16.607980+0000","last_became_peered":"2024-04-02T16:23:16.607980+0000","last_unstale":"2024-04-02T16:23:16.614269+0000","last_undegraded":"2024-04-02T16:23:16.614269+0000","last_fullsized":"2024-04-02T16:23:16.614269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:17.861607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587103+0000","last_change":"2024-04-02T16:23:15.601122+0000","last_active":"2024-04-02T16:23:17.587103+0000","last_peered":"2024-04-02T16:23:17.587103+0000","last_clean":"2024-04-02T16:23:17.587103+0000","last_became_active":"2024-04-02T16:23:15.600811+0000","last_became_peered":"2024-04-02T16:23:15.600811+0000","last_unstale":"2024-04-02T16:23:17.587103+0000","last_undegraded":"2024-04-02T16:23:17.587103+0000","last_fullsized":"2024-04-02T16:23:17.587103+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:22.478731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587143+0000","last_change":"2024-04-02T16:23:16.594813+0000","last_active":"2024-04-02T16:23:17.587143+0000","last_peered":"2024-04-02T16:23:17.587143+0000","last_clean":"2024-04-02T16:23:17.587143+0000","last_became_active":"2024-04-02T16:23:16.594564+0000","last_became_peered":"2024-04-02T16:23:16.594564+0000","last_unstale":"2024-04-02T16:23:17.587143+0000","last_undegraded":"2024-04-02T16:23:17.587143+0000","last_fullsized":"2024-04-02T16:23:17.587143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:51.357191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.608953+0000","last_change":"2024-04-02T16:23:15.599786+0000","last_active":"2024-04-02T16:23:16.608953+0000","last_peered":"2024-04-02T16:23:16.608953+0000","last_clean":"2024-04-02T16:23:16.608953+0000","last_became_active":"2024-04-02T16:23:15.599262+0000","last_became_peered":"2024-04-02T16:23:15.599262+0000","last_unstale":"2024-04-02T16:23:16.608953+0000","last_undegraded":"2024-04-02T16:23:16.608953+0000","last_fullsized":"2024-04-02T16:23:16.608953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:54.455581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587838+0000","last_change":"2024-04-02T16:23:16.602963+0000","last_active":"2024-04-02T16:23:17.587838+0000","last_peered":"2024-04-02T16:23:17.587838+0000","last_clean":"2024-04-02T16:23:17.587838+0000","last_became_active":"2024-04-02T16:23:16.602692+0000","last_became_peered":"2024-04-02T16:23:16.602692+0000","last_unstale":"2024-04-02T16:23:17.587838+0000","last_undegraded":"2024-04-02T16:23:17.587838+0000","last_fullsized":"2024-04-02T16:23:17.587838+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:24:59.438279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.590093+0000","last_change":"2024-04-02T16:23:15.593798+0000","last_active":"2024-04-02T16:23:16.590093+0000","last_peered":"2024-04-02T16:23:16.590093+0000","last_clean":"2024-04-02T16:23:16.590093+0000","last_became_active":"2024-04-02T16:23:15.593534+0000","last_became_peered":"2024-04-02T16:23:15.593534+0000","last_unstale":"2024-04-02T16:23:16.590093+0000","last_undegraded":"2024-04-02T16:23:16.590093+0000","last_fullsized":"2024-04-02T16:23:16.590093+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:51.685925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585983+0000","last_change":"2024-04-02T16:23:16.599801+0000","last_active":"2024-04-02T16:23:17.585983+0000","last_peered":"2024-04-02T16:23:17.585983+0000","last_clean":"2024-04-02T16:23:17.585983+0000","last_became_active":"2024-04-02T16:23:16.599495+0000","last_became_peered":"2024-04-02T16:23:16.599495+0000","last_unstale":"2024-04-02T16:23:17.585983+0000","last_undegraded":"2024-04-02T16:23:17.585983+0000","last_fullsized":"2024-04-02T16:23:17.585983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:33:49.551083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587122+0000","last_change":"2024-04-02T16:23:15.594881+0000","last_active":"2024-04-02T16:23:17.587122+0000","last_peered":"2024-04-02T16:23:17.587122+0000","last_clean":"2024-04-02T16:23:17.587122+0000","last_became_active":"2024-04-02T16:23:15.594521+0000","last_became_peered":"2024-04-02T16:23:15.594521+0000","last_unstale":"2024-04-02T16:23:17.587122+0000","last_undegraded":"2024-04-02T16:23:17.587122+0000","last_fullsized":"2024-04-02T16:23:17.587122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:51.685925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605985+0000","last_change":"2024-04-02T16:23:16.603117+0000","last_active":"2024-04-02T16:23:17.605985+0000","last_peered":"2024-04-02T16:23:17.605985+0000","last_clean":"2024-04-02T16:23:17.605985+0000","last_became_active":"2024-04-02T16:23:16.599775+0000","last_became_peered":"2024-04-02T16:23:16.599775+0000","last_unstale":"2024-04-02T16:23:17.605985+0000","last_undegraded":"2024-04-02T16:23:17.605985+0000","last_fullsized":"2024-04-02T16:23:17.605985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:45.384355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586259+0000","last_change":"2024-04-02T16:23:15.591988+0000","last_active":"2024-04-02T16:23:17.586259+0000","last_peered":"2024-04-02T16:23:17.586259+0000","last_clean":"2024-04-02T16:23:17.586259+0000","last_became_active":"2024-04-02T16:23:15.591579+0000","last_became_peered":"2024-04-02T16:23:15.591579+0000","last_unstale":"2024-04-02T16:23:17.586259+0000","last_undegraded":"2024-04-02T16:23:17.586259+0000","last_fullsized":"2024-04-02T16:23:17.586259+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:25.021708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.604234+0000","last_change":"2024-04-02T16:23:16.598657+0000","last_active":"2024-04-02T16:23:16.604234+0000","last_peered":"2024-04-02T16:23:16.604234+0000","last_clean":"2024-04-02T16:23:16.604234+0000","last_became_active":"2024-04-02T16:23:16.598411+0000","last_became_peered":"2024-04-02T16:23:16.598411+0000","last_unstale":"2024-04-02T16:23:16.604234+0000","last_undegraded":"2024-04-02T16:23:16.604234+0000","last_fullsized":"2024-04-02T16:23:16.604234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:13.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587529+0000","last_change":"2024-04-02T16:23:15.591680+0000","last_active":"2024-04-02T16:23:17.587529+0000","last_peered":"2024-04-02T16:23:17.587529+0000","last_clean":"2024-04-02T16:23:17.587529+0000","last_became_active":"2024-04-02T16:23:15.591398+0000","last_became_peered":"2024-04-02T16:23:15.591398+0000","last_unstale":"2024-04-02T16:23:17.587529+0000","last_undegraded":"2024-04-02T16:23:17.587529+0000","last_fullsized":"2024-04-02T16:23:17.587529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:12.035337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.611112+0000","last_change":"2024-04-02T16:23:16.607312+0000","last_active":"2024-04-02T16:23:16.611112+0000","last_peered":"2024-04-02T16:23:16.611112+0000","last_clean":"2024-04-02T16:23:16.611112+0000","last_became_active":"2024-04-02T16:23:16.607032+0000","last_became_peered":"2024-04-02T16:23:16.607032+0000","last_unstale":"2024-04-02T16:23:16.611112+0000","last_undegraded":"2024-04-02T16:23:16.611112+0000","last_fullsized":"2024-04-02T16:23:16.611112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:59.966073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587107+0000","last_change":"2024-04-02T16:23:15.589106+0000","last_active":"2024-04-02T16:23:17.587107+0000","last_peered":"2024-04-02T16:23:17.587107+0000","last_clean":"2024-04-02T16:23:17.587107+0000","last_became_active":"2024-04-02T16:23:15.588440+0000","last_became_peered":"2024-04-02T16:23:15.588440+0000","last_unstale":"2024-04-02T16:23:17.587107+0000","last_undegraded":"2024-04-02T16:23:17.587107+0000","last_fullsized":"2024-04-02T16:23:17.587107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:43.376692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587671+0000","last_change":"2024-04-02T16:23:16.595513+0000","last_active":"2024-04-02T16:23:17.587671+0000","last_peered":"2024-04-02T16:23:17.587671+0000","last_clean":"2024-04-02T16:23:17.587671+0000","last_became_active":"2024-04-02T16:23:16.594290+0000","last_became_peered":"2024-04-02T16:23:16.594290+0000","last_unstale":"2024-04-02T16:23:17.587671+0000","last_undegraded":"2024-04-02T16:23:17.587671+0000","last_fullsized":"2024-04-02T16:23:17.587671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:23.044403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586070+0000","last_change":"2024-04-02T16:23:15.594529+0000","last_active":"2024-04-02T16:23:17.586070+0000","last_peered":"2024-04-02T16:23:17.586070+0000","last_clean":"2024-04-02T16:23:17.586070+0000","last_became_active":"2024-04-02T16:23:15.594103+0000","last_became_peered":"2024-04-02T16:23:15.594103+0000","last_unstale":"2024-04-02T16:23:17.586070+0000","last_undegraded":"2024-04-02T16:23:17.586070+0000","last_fullsized":"2024-04-02T16:23:17.586070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:57:03.723827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.614286+0000","last_change":"2024-04-02T16:23:16.610013+0000","last_active":"2024-04-02T16:23:16.614286+0000","last_peered":"2024-04-02T16:23:16.614286+0000","last_clean":"2024-04-02T16:23:16.614286+0000","last_became_active":"2024-04-02T16:23:16.609730+0000","last_became_peered":"2024-04-02T16:23:16.609730+0000","last_unstale":"2024-04-02T16:23:16.614286+0000","last_undegraded":"2024-04-02T16:23:16.614286+0000","last_fullsized":"2024-04-02T16:23:16.614286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:46:26.024041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586109+0000","last_change":"2024-04-02T16:23:15.601030+0000","last_active":"2024-04-02T16:23:17.586109+0000","last_peered":"2024-04-02T16:23:17.586109+0000","last_clean":"2024-04-02T16:23:17.586109+0000","last_became_active":"2024-04-02T16:23:15.600768+0000","last_became_peered":"2024-04-02T16:23:15.600768+0000","last_unstale":"2024-04-02T16:23:17.586109+0000","last_undegraded":"2024-04-02T16:23:17.586109+0000","last_fullsized":"2024-04-02T16:23:17.586109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:02:04.853324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585791+0000","last_change":"2024-04-02T16:23:16.602319+0000","last_active":"2024-04-02T16:23:17.585791+0000","last_peered":"2024-04-02T16:23:17.585791+0000","last_clean":"2024-04-02T16:23:17.585791+0000","last_became_active":"2024-04-02T16:23:16.602062+0000","last_became_peered":"2024-04-02T16:23:16.602062+0000","last_unstale":"2024-04-02T16:23:17.585791+0000","last_undegraded":"2024-04-02T16:23:17.585791+0000","last_fullsized":"2024-04-02T16:23:17.585791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:51.357191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.590720+0000","last_change":"2024-04-02T16:23:15.594534+0000","last_active":"2024-04-02T16:23:16.590720+0000","last_peered":"2024-04-02T16:23:16.590720+0000","last_clean":"2024-04-02T16:23:16.590720+0000","last_became_active":"2024-04-02T16:23:15.594233+0000","last_became_peered":"2024-04-02T16:23:15.594233+0000","last_unstale":"2024-04-02T16:23:16.590720+0000","last_undegraded":"2024-04-02T16:23:16.590720+0000","last_fullsized":"2024-04-02T16:23:16.590720+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:12.035337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.615678+0000","last_change":"2024-04-02T16:23:16.612243+0000","last_active":"2024-04-02T16:23:16.615678+0000","last_peered":"2024-04-02T16:23:16.615678+0000","last_clean":"2024-04-02T16:23:16.615678+0000","last_became_active":"2024-04-02T16:23:16.611908+0000","last_became_peered":"2024-04-02T16:23:16.611908+0000","last_unstale":"2024-04-02T16:23:16.615678+0000","last_undegraded":"2024-04-02T16:23:16.615678+0000","last_fullsized":"2024-04-02T16:23:16.615678+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:02:05.855656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.587750+0000","last_change":"2024-04-02T16:23:15.599541+0000","last_active":"2024-04-02T16:23:16.587750+0000","last_peered":"2024-04-02T16:23:16.587750+0000","last_clean":"2024-04-02T16:23:16.587750+0000","last_became_active":"2024-04-02T16:23:15.599251+0000","last_became_peered":"2024-04-02T16:23:15.599251+0000","last_unstale":"2024-04-02T16:23:16.587750+0000","last_undegraded":"2024-04-02T16:23:16.587750+0000","last_fullsized":"2024-04-02T16:23:16.587750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:58.181543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585982+0000","last_change":"2024-04-02T16:23:16.615009+0000","last_active":"2024-04-02T16:23:17.585982+0000","last_peered":"2024-04-02T16:23:17.585982+0000","last_clean":"2024-04-02T16:23:17.585982+0000","last_became_active":"2024-04-02T16:23:16.611215+0000","last_became_peered":"2024-04-02T16:23:16.611215+0000","last_unstale":"2024-04-02T16:23:17.585982+0000","last_undegraded":"2024-04-02T16:23:17.585982+0000","last_fullsized":"2024-04-02T16:23:17.585982+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:16.140068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.609628+0000","last_change":"2024-04-02T16:23:15.599617+0000","last_active":"2024-04-02T16:23:16.609628+0000","last_peered":"2024-04-02T16:23:16.609628+0000","last_clean":"2024-04-02T16:23:16.609628+0000","last_became_active":"2024-04-02T16:23:15.599314+0000","last_became_peered":"2024-04-02T16:23:15.599314+0000","last_unstale":"2024-04-02T16:23:16.609628+0000","last_undegraded":"2024-04-02T16:23:16.609628+0000","last_fullsized":"2024-04-02T16:23:16.609628+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:00:09.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586109+0000","last_change":"2024-04-02T16:23:16.599855+0000","last_active":"2024-04-02T16:23:17.586109+0000","last_peered":"2024-04-02T16:23:17.586109+0000","last_clean":"2024-04-02T16:23:17.586109+0000","last_became_active":"2024-04-02T16:23:16.599580+0000","last_became_peered":"2024-04-02T16:23:16.599580+0000","last_unstale":"2024-04-02T16:23:17.586109+0000","last_undegraded":"2024-04-02T16:23:17.586109+0000","last_fullsized":"2024-04-02T16:23:17.586109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:00.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587241+0000","last_change":"2024-04-02T16:23:15.597637+0000","last_active":"2024-04-02T16:23:17.587241+0000","last_peered":"2024-04-02T16:23:17.587241+0000","last_clean":"2024-04-02T16:23:17.587241+0000","last_became_active":"2024-04-02T16:23:15.597344+0000","last_became_peered":"2024-04-02T16:23:15.597344+0000","last_unstale":"2024-04-02T16:23:17.587241+0000","last_undegraded":"2024-04-02T16:23:17.587241+0000","last_fullsized":"2024-04-02T16:23:17.587241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:20:58.963741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586977+0000","last_change":"2024-04-02T16:23:16.598736+0000","last_active":"2024-04-02T16:23:17.586977+0000","last_peered":"2024-04-02T16:23:17.586977+0000","last_clean":"2024-04-02T16:23:17.586977+0000","last_became_active":"2024-04-02T16:23:16.598448+0000","last_became_peered":"2024-04-02T16:23:16.598448+0000","last_unstale":"2024-04-02T16:23:17.586977+0000","last_undegraded":"2024-04-02T16:23:17.586977+0000","last_fullsized":"2024-04-02T16:23:17.586977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:50.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.588018+0000","last_change":"2024-04-02T16:23:15.594971+0000","last_active":"2024-04-02T16:23:17.588018+0000","last_peered":"2024-04-02T16:23:17.588018+0000","last_clean":"2024-04-02T16:23:17.588018+0000","last_became_active":"2024-04-02T16:23:15.594680+0000","last_became_peered":"2024-04-02T16:23:15.594680+0000","last_unstale":"2024-04-02T16:23:17.588018+0000","last_undegraded":"2024-04-02T16:23:17.588018+0000","last_fullsized":"2024-04-02T16:23:17.588018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:19:22.478731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587787+0000","last_change":"2024-04-02T16:23:16.600865+0000","last_active":"2024-04-02T16:23:17.587787+0000","last_peered":"2024-04-02T16:23:17.587787+0000","last_clean":"2024-04-02T16:23:17.587787+0000","last_became_active":"2024-04-02T16:23:16.600607+0000","last_became_peered":"2024-04-02T16:23:16.600607+0000","last_unstale":"2024-04-02T16:23:17.587787+0000","last_undegraded":"2024-04-02T16:23:17.587787+0000","last_fullsized":"2024-04-02T16:23:17.587787+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:54:53.891976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605464+0000","last_change":"2024-04-02T16:23:15.594551+0000","last_active":"2024-04-02T16:23:17.605464+0000","last_peered":"2024-04-02T16:23:17.605464+0000","last_clean":"2024-04-02T16:23:17.605464+0000","last_became_active":"2024-04-02T16:23:15.594221+0000","last_became_peered":"2024-04-02T16:23:15.594221+0000","last_unstale":"2024-04-02T16:23:17.605464+0000","last_undegraded":"2024-04-02T16:23:17.605464+0000","last_fullsized":"2024-04-02T16:23:17.605464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:42:50.964903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605518+0000","last_change":"2024-04-02T16:23:16.594699+0000","last_active":"2024-04-02T16:23:17.605518+0000","last_peered":"2024-04-02T16:23:17.605518+0000","last_clean":"2024-04-02T16:23:17.605518+0000","last_became_active":"2024-04-02T16:23:16.594435+0000","last_became_peered":"2024-04-02T16:23:16.594435+0000","last_unstale":"2024-04-02T16:23:17.605518+0000","last_undegraded":"2024-04-02T16:23:17.605518+0000","last_fullsized":"2024-04-02T16:23:17.605518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:55.457914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.591807+0000","last_change":"2024-04-02T16:23:15.597593+0000","last_active":"2024-04-02T16:23:16.591807+0000","last_peered":"2024-04-02T16:23:16.591807+0000","last_clean":"2024-04-02T16:23:16.591807+0000","last_became_active":"2024-04-02T16:23:15.597295+0000","last_became_peered":"2024-04-02T16:23:15.597295+0000","last_unstale":"2024-04-02T16:23:16.591807+0000","last_undegraded":"2024-04-02T16:23:16.591807+0000","last_fullsized":"2024-04-02T16:23:16.591807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:43.376692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.610775+0000","last_change":"2024-04-02T16:23:16.604883+0000","last_active":"2024-04-02T16:23:16.610775+0000","last_peered":"2024-04-02T16:23:16.610775+0000","last_clean":"2024-04-02T16:23:16.610775+0000","last_became_active":"2024-04-02T16:23:16.604570+0000","last_became_peered":"2024-04-02T16:23:16.604570+0000","last_unstale":"2024-04-02T16:23:16.610775+0000","last_undegraded":"2024-04-02T16:23:16.610775+0000","last_fullsized":"2024-04-02T16:23:16.610775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:57:04.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605469+0000","last_change":"2024-04-02T16:23:15.598860+0000","last_active":"2024-04-02T16:23:17.605469+0000","last_peered":"2024-04-02T16:23:17.605469+0000","last_clean":"2024-04-02T16:23:17.605469+0000","last_became_active":"2024-04-02T16:23:15.598543+0000","last_became_peered":"2024-04-02T16:23:15.598543+0000","last_unstale":"2024-04-02T16:23:17.605469+0000","last_undegraded":"2024-04-02T16:23:17.605469+0000","last_fullsized":"2024-04-02T16:23:17.605469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:01:35.512859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.611080+0000","last_change":"2024-04-02T16:23:16.607205+0000","last_active":"2024-04-02T16:23:16.611080+0000","last_peered":"2024-04-02T16:23:16.611080+0000","last_clean":"2024-04-02T16:23:16.611080+0000","last_became_active":"2024-04-02T16:23:16.606406+0000","last_became_peered":"2024-04-02T16:23:16.606406+0000","last_unstale":"2024-04-02T16:23:16.611080+0000","last_undegraded":"2024-04-02T16:23:16.611080+0000","last_fullsized":"2024-04-02T16:23:16.611080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:44.379025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605342+0000","last_change":"2024-04-02T16:23:15.594276+0000","last_active":"2024-04-02T16:23:17.605342+0000","last_peered":"2024-04-02T16:23:17.605342+0000","last_clean":"2024-04-02T16:23:17.605342+0000","last_became_active":"2024-04-02T16:23:15.593980+0000","last_became_peered":"2024-04-02T16:23:15.593980+0000","last_unstale":"2024-04-02T16:23:17.605342+0000","last_undegraded":"2024-04-02T16:23:17.605342+0000","last_fullsized":"2024-04-02T16:23:17.605342+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:41.665575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587208+0000","last_change":"2024-04-02T16:23:16.598645+0000","last_active":"2024-04-02T16:23:17.587208+0000","last_peered":"2024-04-02T16:23:17.587208+0000","last_clean":"2024-04-02T16:23:17.587208+0000","last_became_active":"2024-04-02T16:23:16.598375+0000","last_became_peered":"2024-04-02T16:23:16.598375+0000","last_unstale":"2024-04-02T16:23:17.587208+0000","last_undegraded":"2024-04-02T16:23:17.587208+0000","last_fullsized":"2024-04-02T16:23:17.587208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:59:16.140068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.593039+0000","last_change":"2024-04-02T16:23:15.594935+0000","last_active":"2024-04-02T16:23:16.593039+0000","last_peered":"2024-04-02T16:23:16.593039+0000","last_clean":"2024-04-02T16:23:16.593039+0000","last_became_active":"2024-04-02T16:23:15.594618+0000","last_became_peered":"2024-04-02T16:23:15.594618+0000","last_unstale":"2024-04-02T16:23:16.593039+0000","last_undegraded":"2024-04-02T16:23:16.593039+0000","last_fullsized":"2024-04-02T16:23:16.593039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:44.382023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586135+0000","last_change":"2024-04-02T16:23:16.597788+0000","last_active":"2024-04-02T16:23:17.586135+0000","last_peered":"2024-04-02T16:23:17.586135+0000","last_clean":"2024-04-02T16:23:17.586135+0000","last_became_active":"2024-04-02T16:23:16.597423+0000","last_became_peered":"2024-04-02T16:23:16.597423+0000","last_unstale":"2024-04-02T16:23:17.586135+0000","last_undegraded":"2024-04-02T16:23:17.586135+0000","last_fullsized":"2024-04-02T16:23:17.586135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:59:33.161559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605529+0000","last_change":"2024-04-02T16:23:15.600429+0000","last_active":"2024-04-02T16:23:17.605529+0000","last_peered":"2024-04-02T16:23:17.605529+0000","last_clean":"2024-04-02T16:23:17.605529+0000","last_became_active":"2024-04-02T16:23:15.600140+0000","last_became_peered":"2024-04-02T16:23:15.600140+0000","last_unstale":"2024-04-02T16:23:17.605529+0000","last_undegraded":"2024-04-02T16:23:17.605529+0000","last_fullsized":"2024-04-02T16:23:17.605529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:32:19.499561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.606105+0000","last_change":"2024-04-02T16:23:16.601486+0000","last_active":"2024-04-02T16:23:16.606105+0000","last_peered":"2024-04-02T16:23:16.606105+0000","last_clean":"2024-04-02T16:23:16.606105+0000","last_became_active":"2024-04-02T16:23:16.601199+0000","last_became_peered":"2024-04-02T16:23:16.601199+0000","last_unstale":"2024-04-02T16:23:16.606105+0000","last_undegraded":"2024-04-02T16:23:16.606105+0000","last_fullsized":"2024-04-02T16:23:16.606105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:17.861607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587423+0000","last_change":"2024-04-02T16:23:15.588940+0000","last_active":"2024-04-02T16:23:17.587423+0000","last_peered":"2024-04-02T16:23:17.587423+0000","last_clean":"2024-04-02T16:23:17.587423+0000","last_became_active":"2024-04-02T16:23:15.588413+0000","last_became_peered":"2024-04-02T16:23:15.588413+0000","last_unstale":"2024-04-02T16:23:17.587423+0000","last_undegraded":"2024-04-02T16:23:17.587423+0000","last_fullsized":"2024-04-02T16:23:17.587423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:45.252611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.613690+0000","last_change":"2024-04-02T16:23:16.611229+0000","last_active":"2024-04-02T16:23:16.613690+0000","last_peered":"2024-04-02T16:23:16.613690+0000","last_clean":"2024-04-02T16:23:16.613690+0000","last_became_active":"2024-04-02T16:23:16.610937+0000","last_became_peered":"2024-04-02T16:23:16.610937+0000","last_unstale":"2024-04-02T16:23:16.613690+0000","last_undegraded":"2024-04-02T16:23:16.613690+0000","last_fullsized":"2024-04-02T16:23:16.613690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:57.683621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.605346+0000","last_change":"2024-04-02T16:23:15.593263+0000","last_active":"2024-04-02T16:23:17.605346+0000","last_peered":"2024-04-02T16:23:17.605346+0000","last_clean":"2024-04-02T16:23:17.605346+0000","last_became_active":"2024-04-02T16:23:15.592965+0000","last_became_peered":"2024-04-02T16:23:15.592965+0000","last_unstale":"2024-04-02T16:23:17.605346+0000","last_undegraded":"2024-04-02T16:23:17.605346+0000","last_fullsized":"2024-04-02T16:23:17.605346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.132590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.586282+0000","last_change":"2024-04-02T16:23:16.611900+0000","last_active":"2024-04-02T16:23:17.586282+0000","last_peered":"2024-04-02T16:23:17.586282+0000","last_clean":"2024-04-02T16:23:17.586282+0000","last_became_active":"2024-04-02T16:23:16.606097+0000","last_became_peered":"2024-04-02T16:23:16.606097+0000","last_unstale":"2024-04-02T16:23:17.586282+0000","last_undegraded":"2024-04-02T16:23:17.586282+0000","last_fullsized":"2024-04-02T16:23:17.586282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:25:17.719148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.596371+0000","last_change":"2024-04-02T16:23:15.598453+0000","last_active":"2024-04-02T16:23:16.596371+0000","last_peered":"2024-04-02T16:23:16.596371+0000","last_clean":"2024-04-02T16:23:16.596371+0000","last_became_active":"2024-04-02T16:23:15.598170+0000","last_became_peered":"2024-04-02T16:23:15.598170+0000","last_unstale":"2024-04-02T16:23:16.596371+0000","last_undegraded":"2024-04-02T16:23:16.596371+0000","last_fullsized":"2024-04-02T16:23:16.596371+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:54.455581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.614246+0000","last_change":"2024-04-02T16:23:16.607993+0000","last_active":"2024-04-02T16:23:16.614246+0000","last_peered":"2024-04-02T16:23:16.614246+0000","last_clean":"2024-04-02T16:23:16.614246+0000","last_became_active":"2024-04-02T16:23:16.607667+0000","last_became_peered":"2024-04-02T16:23:16.607667+0000","last_unstale":"2024-04-02T16:23:16.614246+0000","last_undegraded":"2024-04-02T16:23:16.614246+0000","last_fullsized":"2024-04-02T16:23:16.614246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:46.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.596713+0000","last_change":"2024-04-02T16:23:15.600399+0000","last_active":"2024-04-02T16:23:16.596713+0000","last_peered":"2024-04-02T16:23:16.596713+0000","last_clean":"2024-04-02T16:23:16.596713+0000","last_became_active":"2024-04-02T16:23:15.599981+0000","last_became_peered":"2024-04-02T16:23:15.599981+0000","last_unstale":"2024-04-02T16:23:16.596713+0000","last_undegraded":"2024-04-02T16:23:16.596713+0000","last_fullsized":"2024-04-02T16:23:16.596713+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:17:44.382023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.602508+0000","last_change":"2024-04-02T16:23:16.598611+0000","last_active":"2024-04-02T16:23:16.602508+0000","last_peered":"2024-04-02T16:23:16.602508+0000","last_clean":"2024-04-02T16:23:16.602508+0000","last_became_active":"2024-04-02T16:23:16.598335+0000","last_became_peered":"2024-04-02T16:23:16.598335+0000","last_unstale":"2024-04-02T16:23:16.602508+0000","last_undegraded":"2024-04-02T16:23:16.602508+0000","last_fullsized":"2024-04-02T16:23:16.602508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:26:46.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.592877+0000","last_change":"2024-04-02T16:23:15.600550+0000","last_active":"2024-04-02T16:23:16.592877+0000","last_peered":"2024-04-02T16:23:16.592877+0000","last_clean":"2024-04-02T16:23:16.592877+0000","last_became_active":"2024-04-02T16:23:15.600266+0000","last_became_peered":"2024-04-02T16:23:15.600266+0000","last_unstale":"2024-04-02T16:23:16.592877+0000","last_undegraded":"2024-04-02T16:23:16.592877+0000","last_fullsized":"2024-04-02T16:23:16.592877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:18:54.455581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587618+0000","last_change":"2024-04-02T16:23:16.600255+0000","last_active":"2024-04-02T16:23:17.587618+0000","last_peered":"2024-04-02T16:23:17.587618+0000","last_clean":"2024-04-02T16:23:17.587618+0000","last_became_active":"2024-04-02T16:23:16.599861+0000","last_became_peered":"2024-04-02T16:23:16.599861+0000","last_unstale":"2024-04-02T16:23:17.587618+0000","last_undegraded":"2024-04-02T16:23:17.587618+0000","last_fullsized":"2024-04-02T16:23:17.587618+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:32.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587536+0000","last_change":"2024-04-02T16:23:15.597125+0000","last_active":"2024-04-02T16:23:17.587536+0000","last_peered":"2024-04-02T16:23:17.587536+0000","last_clean":"2024-04-02T16:23:17.587536+0000","last_became_active":"2024-04-02T16:23:15.596724+0000","last_became_peered":"2024-04-02T16:23:15.596724+0000","last_unstale":"2024-04-02T16:23:17.587536+0000","last_undegraded":"2024-04-02T16:23:17.587536+0000","last_fullsized":"2024-04-02T16:23:17.587536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:43.376692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.603506+0000","last_change":"2024-04-02T16:23:16.598737+0000","last_active":"2024-04-02T16:23:16.603506+0000","last_peered":"2024-04-02T16:23:16.603506+0000","last_clean":"2024-04-02T16:23:16.603506+0000","last_became_active":"2024-04-02T16:23:16.598382+0000","last_became_peered":"2024-04-02T16:23:16.598382+0000","last_unstale":"2024-04-02T16:23:16.603506+0000","last_undegraded":"2024-04-02T16:23:16.603506+0000","last_fullsized":"2024-04-02T16:23:16.603506+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:00:44.379025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.585724+0000","last_change":"2024-04-02T16:23:15.592737+0000","last_active":"2024-04-02T16:23:17.585724+0000","last_peered":"2024-04-02T16:23:17.585724+0000","last_clean":"2024-04-02T16:23:17.585724+0000","last_became_active":"2024-04-02T16:23:15.592438+0000","last_became_peered":"2024-04-02T16:23:15.592438+0000","last_unstale":"2024-04-02T16:23:17.585724+0000","last_undegraded":"2024-04-02T16:23:17.585724+0000","last_fullsized":"2024-04-02T16:23:17.585724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:03:56.681288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T16:23:17.587359+0000","last_change":"2024-04-02T16:23:16.606633+0000","last_active":"2024-04-02T16:23:17.587359+0000","last_peered":"2024-04-02T16:23:17.587359+0000","last_clean":"2024-04-02T16:23:17.587359+0000","last_became_active":"2024-04-02T16:23:16.606368+0000","last_became_peered":"2024-04-02T16:23:16.606368+0000","last_unstale":"2024-04-02T16:23:17.587359+0000","last_undegraded":"2024-04-02T16:23:17.587359+0000","last_fullsized":"2024-04-02T16:23:17.587359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:15.573379+0000","last_clean_scrub_stamp":"2024-04-02T16:23:15.573379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:59:33.161559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.593510+0000","last_change":"2024-04-02T16:23:15.597318+0000","last_active":"2024-04-02T16:23:16.593510+0000","last_peered":"2024-04-02T16:23:16.593510+0000","last_clean":"2024-04-02T16:23:16.593510+0000","last_became_active":"2024-04-02T16:23:15.597016+0000","last_became_peered":"2024-04-02T16:23:15.597016+0000","last_unstale":"2024-04-02T16:23:16.593510+0000","last_undegraded":"2024-04-02T16:23:16.593510+0000","last_fullsized":"2024-04-02T16:23:16.593510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:23:14.571047+0000","last_clean_scrub_stamp":"2024-04-02T16:23:14.571047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:20:51.685925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-02T16:23:16.593131+0000","last_change":"2024-04-02T16:21:49.851609+0000","last_active":"2024-04-02T16:23:16.593131+0000","last_peered":"2024-04-02T16:23:16.593131+0000","last_clean":"2024-04-02T16:23:16.593131+0000","last_became_active":"2024-04-02T16:21:49.850564+0000","last_became_peered":"2024-04-02T16:21:49.850564+0000","last_unstale":"2024-04-02T16:23:16.593131+0000","last_undegraded":"2024-04-02T16:23:16.593131+0000","last_fullsized":"2024-04-02T16:23:16.593131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:21:48.692321+0000","last_clean_scrub_stamp":"2024-04-02T16:21:48.692321+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.825754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":191225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":198981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":781045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:23:19.968 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T16:23:19.968 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-02T16:23:20.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.114+0000 7f7ca10d5640 1 Processor -- start 2024-04-02T16:23:20.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.114+0000 7f7ca10d5640 1 -- start start 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7ca10d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 0x7f7c9c0d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7ca10d5640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0d2f30 con 0x7f7c9c15e7b0 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7ca10d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0cf000 con 0x7f7c9c0c7e30 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7ca10d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c0cf2e0 con 0x7f7c9c15e3d0 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c9b59e640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f7c9c15e7b0 0x7f7c9c0cfae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42482/0 (socket says 172.21.15.50:42482) 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c9b59e640 1 -- 172.21.15.50:0/1255128624 learned_addr learned my addr 172.21.15.50:0/1255128624 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c9ad9d640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 0x7f7c9c0d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3823763733 0 0) 0x7f7c9c0d2f30 con 0x7f7c9c15e7b0 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c84003520 con 0x7f7c9c15e7b0 2024-04-02T16:23:20.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 431145124 0 0) 0x7f7c9c0cf000 con 0x7f7c9c0c7e30 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c9c0d2f30 con 0x7f7c9c0c7e30 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 59286636 0 0) 0x7f7c84003520 con 0x7f7c9c15e7b0 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 msgr2=0x7f7c9c0d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 0x7f7c9c0d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 >> v1:172.21.15.188:6789/0 conn(0x7f7c9c0c7e30 legacy=0x7f7c9c0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0cf5c0 con 0x7f7c9c15e7b0 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f7c8c0019f0 con 0x7f7c9c15e7b0 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 >> v1:172.21.15.50:6789/0 conn(0x7f7c9c15e7b0 legacy=0x7f7c9c0cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c84003b70 0x7f7c84003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.115+0000 7f7c99d9b640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 0x7f7c84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c99d9b640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c84004d40 0x7f7c840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c0cf2e0 con 0x7f7c84004d40 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8400ba70 con 0x7f7c84003b70 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8400bc20 con 0x7f7c9c15e3d0 2024-04-02T16:23:20.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9b59e640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c84004d40 0x7f7c840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9ad9d640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c84003b70 0x7f7c84003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9a59c640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 0x7f7c84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9b59e640 1 -- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 msgr2=0x7f7c84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9b59e640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c15e3d0 0x7f7c84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9b59e640 1 -- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c84003b70 msgr2=0x7f7c84003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9b59e640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c84003b70 0x7f7c84003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.116+0000 7f7c9b59e640 1 -- 172.21.15.50:0/1255128624 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c84003520 con 0x7f7c84004d40 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7c9b59e640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c84004d40 0x7f7c840051c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0b5ae0 tx=0x7f7c84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfc4056ffd7e107 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00a430 con 0x7f7c84004d40 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c8c034630 con 0x7f7c84004d40 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7c99d9b640 1 -- 172.21.15.50:0/1255128624 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c034a20 con 0x7f7c84004d40 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7ca10d5640 1 -- 172.21.15.50:0/1255128624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c84004d40 msgr2=0x7f7c840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7ca10d5640 1 --2- 172.21.15.50:0/1255128624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c84004d40 0x7f7c840051c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0b5ae0 tx=0x7f7c84005830 comp rx=0 tx=0).stop 2024-04-02T16:23:20.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7c9b59e640 1 -- 172.21.15.50:0/1255128624 reap_dead start 2024-04-02T16:23:20.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7ca10d5640 1 -- 172.21.15.50:0/1255128624 shutdown_connections 2024-04-02T16:23:20.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7ca10d5640 1 -- 172.21.15.50:0/1255128624 >> 172.21.15.50:0/1255128624 conn(0x7f7c9c08dc90 msgr2=0x7f7c9c09d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:20.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7ca10d5640 1 -- 172.21.15.50:0/1255128624 shutdown_connections 2024-04-02T16:23:20.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.117+0000 7f7ca10d5640 1 -- 172.21.15.50:0/1255128624 wait complete. 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.119+0000 7f7ca10d5640 1 Processor -- start 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.119+0000 7f7ca10d5640 1 -- start start 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.119+0000 7f7ca10d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c9c1f7e40 0x7f7c9c158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7ca10d5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c9c158d80 0x7f7c9c1443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7ca10d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c144920 0x7f7c9c15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7ca10d5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c0d2a00 con 0x7f7c9c158d80 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7ca10d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c0d2880 con 0x7f7c9c1f7e40 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7ca10d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c0d2b80 con 0x7f7c9c144920 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c9c158d80 0x7f7c9c1443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c9c158d80 0x7f7c9c1443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41022/0 (socket says 172.21.15.50:41022) 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 -- 172.21.15.50:0/3817529061 learned_addr learned my addr 172.21.15.50:0/3817529061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:20.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9b59e640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c144920 0x7f7c9c15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9ad9d640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c9c1f7e40 0x7f7c9c158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 -- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c144920 msgr2=0x7f7c9c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c144920 0x7f7c9c15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 -- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c9c1f7e40 msgr2=0x7f7c9c158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c9c1f7e40 0x7f7c9c158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 -- 172.21.15.50:0/3817529061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c145150 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c9a59c640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c9c158d80 0x7f7c9c1443e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000f7a0 tx=0x7f7c9000fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c837fe640 1 -- 172.21.15.50:0/3817529061 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900050d0 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c837fe640 1 -- 172.21.15.50:0/3817529061 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c90018070 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c1e7f50 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7c837fe640 1 -- 172.21.15.50:0/3817529061 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000d6d0 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.120+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c1e8510 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.122+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c075760 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.122+0000 7f7c837fe640 1 -- 172.21.15.50:0/3817529061 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c9000d870 con 0x7f7c9c158d80 2024-04-02T16:23:20.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.122+0000 7f7c837fe640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c8806c920 0x7f7c8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.122+0000 7f7c837fe640 1 -- 172.21.15.50:0/3817529061 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7c9008ba60 con 0x7f7c9c158d80 2024-04-02T16:23:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.122+0000 7f7c9ad9d640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c8806c920 0x7f7c8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.123+0000 7f7c9ad9d640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c8806c920 0x7f7c8806ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0ce930 tx=0x7f7c94009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:20.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.127+0000 7f7c837fe640 1 -- 172.21.15.50:0/3817529061 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7c90059d00 con 0x7f7c9c158d80 2024-04-02T16:23:20.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.382+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7c9c162780 con 0x7f7c9c158d80 2024-04-02T16:23:20.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.384+0000 7f7c837fe640 1 -- 172.21.15.50:0/3817529061 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f7c90059600 con 0x7f7c9c158d80 2024-04-02T16:23:20.384 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-04-02T16:23:20.384 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBxMAxmDpViBRAAI+QEeaTzgk7SzJG7sathbQ== 2024-04-02T16:23:20.384 INFO:teuthology.orchestra.run.smithi050.stdout: caps mds = "allow" 2024-04-02T16:23:20.384 INFO:teuthology.orchestra.run.smithi050.stdout: caps mon = "allow r" 2024-04-02T16:23:20.385 INFO:teuthology.orchestra.run.smithi050.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-02T16:23:20.385 INFO:teuthology.orchestra.run.smithi050.stderr:updated caps for client.0 2024-04-02T16:23:20.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c8806c920 msgr2=0x7f7c8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c8806c920 0x7f7c8806ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0ce930 tx=0x7f7c94009190 comp rx=0 tx=0).stop 2024-04-02T16:23:20.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c9c158d80 msgr2=0x7f7c9c1443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c9c158d80 0x7f7c9c1443e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000f7a0 tx=0x7f7c9000fba0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 shutdown_connections 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c9c144920 0x7f7c9c15ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f7c8806c920 0x7f7c8806ede0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c9c158d80 0x7f7c9c1443e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 --2- 172.21.15.50:0/3817529061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c9c1f7e40 0x7f7c9c158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 >> 172.21.15.50:0/3817529061 conn(0x7f7c9c08dc90 msgr2=0x7f7c9c0d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 shutdown_connections 2024-04-02T16:23:20.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.386+0000 7f7ca10d5640 1 -- 172.21.15.50:0/3817529061 wait complete. 2024-04-02T16:23:20.397 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:23:20.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.499+0000 7f8ab099d640 1 Processor -- start 2024-04-02T16:23:20.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.499+0000 7f8ab099d640 1 -- start start 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8ab099d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8aac0d1130 0x7f8aac0d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8ab099d640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aac0d1e10 con 0x7f8aac0cfa10 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8ab099d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aac0d20f0 con 0x7f8aac15e420 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8ab099d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8aac15e800 con 0x7f8aac0d1130 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aabfff640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f8aac0cfa10 0x7f8aac0cfe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42488/0 (socket says 172.21.15.50:42488) 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aabfff640 1 -- 172.21.15.50:0/396138281 learned_addr learned my addr 172.21.15.50:0/396138281 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194299169 0 0) 0x7f8aac0d1e10 con 0x7f8aac0cfa10 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a98003520 con 0x7f8aac0cfa10 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aa3fff640 1 --2- 172.21.15.50:0/396138281 >> v2:172.21.15.188:3300/0 conn(0x7f8aac0d1130 0x7f8aac0d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 73317047 0 0) 0x7f8a98003520 con 0x7f8aac0cfa10 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 >> v2:172.21.15.188:3300/0 conn(0x7f8aac0d1130 msgr2=0x7f8aac0d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 --2- 172.21.15.50:0/396138281 >> v2:172.21.15.188:3300/0 conn(0x7f8aac0d1130 0x7f8aac0d2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 >> v1:172.21.15.188:6789/0 conn(0x7f8aac15e420 legacy=0x7f8aac0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac0d23d0 con 0x7f8aac0cfa10 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.500+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f8aa40019f0 con 0x7f8aac0cfa10 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 >> v1:172.21.15.50:6789/0 conn(0x7f8aac0cfa10 legacy=0x7f8aac0cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aaaffd640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a980039b0 0x7f8a98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aaaffd640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a98004800 0x7f8a98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aaaffd640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8a9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac15e800 con 0x7f8a980051c0 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9800c060 con 0x7f8a98004800 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9800c260 con 0x7f8a980039b0 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aabfff640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8a9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aab7fe640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a980039b0 0x7f8a98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aa3fff640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a98004800 0x7f8a98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aabfff640 1 -- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a980039b0 msgr2=0x7f8a98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:20.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aabfff640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a980039b0 0x7f8a98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aabfff640 1 -- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a98004800 msgr2=0x7f8a98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:20.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aabfff640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a98004800 0x7f8a98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.501+0000 7f8aabfff640 1 -- 172.21.15.50:0/396138281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a98003520 con 0x7f8a980051c0 2024-04-02T16:23:20.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8aabfff640 1 -- 172.21.15.50:0/396138281 reap_dead start 2024-04-02T16:23:20.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8aabfff640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8a9800b9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400a860 tx=0x7f8a98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822fa18c8171e932 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:20.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400a480 con 0x7f8a980051c0 2024-04-02T16:23:20.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa4034630 con 0x7f8a980051c0 2024-04-02T16:23:20.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8aaaffd640 1 -- 172.21.15.50:0/396138281 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4034a20 con 0x7f8a980051c0 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8ab099d640 1 -- 172.21.15.50:0/396138281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 msgr2=0x7f8a9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8ab099d640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8a9800b9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400a860 tx=0x7f8a98005830 comp rx=0 tx=0).stop 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8ab099d640 1 -- 172.21.15.50:0/396138281 shutdown_connections 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8ab099d640 1 --2- 172.21.15.50:0/396138281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8a9800b9c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8ab099d640 1 -- 172.21.15.50:0/396138281 >> 172.21.15.50:0/396138281 conn(0x7f8aac09d410 msgr2=0x7f8aac0d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.502+0000 7f8ab099d640 1 -- 172.21.15.50:0/396138281 shutdown_connections 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.503+0000 7f8ab099d640 1 -- 172.21.15.50:0/396138281 wait complete. 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 Processor -- start 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 -- start start 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8aac15d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aac1f3ff0 0x7f8aac0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8aac0cd7c0 0x7f8aac0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac0d2eb0 con 0x7f8a980051c0 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac0d2990 con 0x7f8aac1f3ff0 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8ab099d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8aac0d3030 con 0x7f8aac0cd7c0 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8aab7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8aac15d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8aab7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8aac15d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41040/0 (socket says 172.21.15.50:41040) 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.504+0000 7f8aab7fe640 1 -- 172.21.15.50:0/247483671 learned_addr learned my addr 172.21.15.50:0/247483671 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aabfff640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8aac0cd7c0 0x7f8aac0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aa3fff640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aac1f3ff0 0x7f8aac0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aab7fe640 1 -- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8aac0cd7c0 msgr2=0x7f8aac0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aab7fe640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8aac0cd7c0 0x7f8aac0cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aab7fe640 1 -- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aac1f3ff0 msgr2=0x7f8aac0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aab7fe640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aac1f3ff0 0x7f8aac0cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aab7fe640 1 -- 172.21.15.50:0/247483671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac0cba70 con 0x7f8a980051c0 2024-04-02T16:23:20.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aab7fe640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8aac15d430 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00f620 tx=0x7f8a9c00fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aa8ff9640 1 -- 172.21.15.50:0/247483671 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0052e0 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aa8ff9640 1 -- 172.21.15.50:0/247483671 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9c018070 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac0cbd50 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8aa8ff9640 1 -- 172.21.15.50:0/247483671 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c003810 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.505+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac0cc310 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.506+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac04f940 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.507+0000 7f8aa8ff9640 1 -- 172.21.15.50:0/247483671 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a9c0039b0 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.507+0000 7f8aa8ff9640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8a8c06c920 0x7f8a8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.507+0000 7f8aa8ff9640 1 -- 172.21.15.50:0/247483671 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f8a9c08b8f0 con 0x7f8a980051c0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.507+0000 7f8aa3fff640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8a8c06c920 0x7f8a8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:20.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.508+0000 7f8aa3fff640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8a8c06c920 0x7f8a8c06ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0ccb10 tx=0x7f8a94009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:20.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.511+0000 7f8aa8ff9640 1 -- 172.21.15.50:0/247483671 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8a9c059c20 con 0x7f8a980051c0 2024-04-02T16:23:20.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.738+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8aac08e1c0 con 0x7f8a980051c0 2024-04-02T16:23:20.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.738+0000 7f8aa8ff9640 1 -- 172.21.15.50:0/247483671 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f8a9c059520 con 0x7f8a980051c0 2024-04-02T16:23:20.739 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:20.739 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/4131511681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":4131511681},{"type":"v1","addr":"172.21.15.188:6837","nonce":4131511681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1831814256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1831814256},{"type":"v1","addr":"172.21.15.188:6836","nonce":1831814256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6834/1153018762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1153018762},{"type":"v1","addr":"172.21.15.50:6834","nonce":1153018762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:23:15.957913+0000","modified":"2024-04-02T16:23:17.958571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5382":{"gid":5382,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6835/3098609271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6833","nonce":3098609271},{"type":"v1","addr":"172.21.15.50:6835","nonce":3098609271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5382,"qdb_cluster":[5382]},"id":2}]} 2024-04-02T16:23:20.739 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 16 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.740+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8a8c06c920 msgr2=0x7f8a8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.740+0000 7f8ab099d640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8a8c06c920 0x7f8a8c06ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0ccb10 tx=0x7f8a94009190 comp rx=0 tx=0).stop 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.740+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 msgr2=0x7f8aac15d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.740+0000 7f8ab099d640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8aac15d430 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00f620 tx=0x7f8a9c00fa20 comp rx=0 tx=0).stop 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 shutdown_connections 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8aac0cd7c0 0x7f8aac0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aac1f3ff0 0x7f8aac0cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8a8c06c920 0x7f8a8c06ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 --2- 172.21.15.50:0/247483671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a980051c0 0x7f8aac15d430 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:20.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 >> 172.21.15.50:0/247483671 conn(0x7f8aac09d410 msgr2=0x7f8aac0cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:20.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 shutdown_connections 2024-04-02T16:23:20.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:20.741+0000 7f8ab099d640 1 -- 172.21.15.50:0/247483671 wait complete. 2024-04-02T16:23:20.858 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T16:23:20.858+0000 7fcf2863b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T16:23:21.155 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T16:23:15.957913+0000', 'modified': '2024-04-02T16:23:17.958571+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5382}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5382': {'gid': 5382, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6835/3098609271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6833', 'nonce': 3098609271}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 3098609271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5382, 'qdb_cluster': [5382]} 2024-04-02T16:23:21.155 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T16:23:21.155 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-02T16:23:21.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.262+0000 7f36d367b640 1 Processor -- start 2024-04-02T16:23:21.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.262+0000 7f36d367b640 1 -- start start 2024-04-02T16:23:21.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36d367b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 0x7f36cc0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36d367b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36cc15c700 con 0x7f36cc15c1f0 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36d367b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36cc15c9e0 con 0x7f36cc0b6120 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36d367b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc06fb40 con 0x7f36cc0d0420 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36d1417640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f36cc15c1f0 0x7f36cc06f430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:42500/0 (socket says 172.21.15.50:42500) 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36d1417640 1 -- 172.21.15.50:0/117242487 learned_addr learned my addr 172.21.15.50:0/117242487 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36d0c16640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 0x7f36cc0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194747680 0 0) 0x7f36cc15c700 con 0x7f36cc15c1f0 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36b8003520 con 0x7f36cc15c1f0 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1508363524 0 0) 0x7f36b8003520 con 0x7f36cc15c1f0 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 msgr2=0x7f36cc0b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 0x7f36cc0b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 >> v1:172.21.15.188:6789/0 conn(0x7f36cc0b6120 legacy=0x7f36cc15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc15ccc0 con 0x7f36cc15c1f0 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.263+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7f36c8001670 con 0x7f36cc15c1f0 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 >> v1:172.21.15.50:6789/0 conn(0x7f36cc15c1f0 legacy=0x7f36cc06f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36c3fff640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b80039e0 0x7f36b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36c3fff640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36b8004800 0x7f36b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36c3fff640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 0x7f36b8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc06fb40 con 0x7f36b8004800 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36b800baf0 con 0x7f36b80039e0 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36b800bcf0 con 0x7f36cc0d0420 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d0c16640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36b8004800 0x7f36b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d1417640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b80039e0 0x7f36b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d1c18640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 0x7f36b8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d0c16640 1 -- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 msgr2=0x7f36b8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d0c16640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 0x7f36b8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d0c16640 1 -- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b80039e0 msgr2=0x7f36b8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d0c16640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b80039e0 0x7f36b8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d0c16640 1 -- 172.21.15.50:0/117242487 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b8003520 con 0x7f36b8004800 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.264+0000 7f36d1c18640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc0d0420 0x7f36b8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36d0c16640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36b8004800 0x7f36b8004c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009a90 tx=0x7f36b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b01717d4bb109c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc039070 con 0x7f36b8004800 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc02c680 con 0x7f36b8004800 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36c3fff640 1 -- 172.21.15.50:0/117242487 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc009690 con 0x7f36b8004800 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36d367b640 1 -- 172.21.15.50:0/117242487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36b8004800 msgr2=0x7f36b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36d367b640 1 --2- 172.21.15.50:0/117242487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36b8004800 0x7f36b8004c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009a90 tx=0x7f36b8005830 comp rx=0 tx=0).stop 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36d1c18640 1 -- 172.21.15.50:0/117242487 reap_dead start 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36d367b640 1 -- 172.21.15.50:0/117242487 shutdown_connections 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36d367b640 1 -- 172.21.15.50:0/117242487 >> 172.21.15.50:0/117242487 conn(0x7f36cc072c10 msgr2=0x7f36cc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.265+0000 7f36d367b640 1 -- 172.21.15.50:0/117242487 shutdown_connections 2024-04-02T16:23:21.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.266+0000 7f36d367b640 1 -- 172.21.15.50:0/117242487 wait complete. 2024-04-02T16:23:21.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 Processor -- start 2024-04-02T16:23:21.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 -- start start 2024-04-02T16:23:21.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b8004800 0x7f36cc153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc1e27c0 0x7f36cc13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36cc14e030 0x7f36cc14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc15d930 con 0x7f36cc14e030 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc15d7b0 con 0x7f36b8004800 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d367b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc0b5be0 con 0x7f36cc1e27c0 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d1c18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36cc14e030 0x7f36cc14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.267+0000 7f36d1c18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36cc14e030 0x7f36cc14e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41052/0 (socket says 172.21.15.50:41052) 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1c18640 1 -- 172.21.15.50:0/1258656252 learned_addr learned my addr 172.21.15.50:0/1258656252 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d0c16640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc1e27c0 0x7f36cc13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1417640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b8004800 0x7f36cc153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1c18640 1 -- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc1e27c0 msgr2=0x7f36cc13e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1c18640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc1e27c0 0x7f36cc13e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1c18640 1 -- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b8004800 msgr2=0x7f36cc153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1c18640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b8004800 0x7f36cc153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1c18640 1 -- 172.21.15.50:0/1258656252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc1e2c40 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d0c16640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc1e27c0 0x7f36cc13e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1417640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b8004800 0x7f36cc153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d1c18640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36cc14e030 0x7f36cc14e4b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f36cc06f190 tx=0x7f36c400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36c1ffb640 1 -- 172.21.15.50:0/1258656252 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4006b50 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36c1ffb640 1 -- 172.21.15.50:0/1258656252 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c4006d10 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36c1ffb640 1 -- 172.21.15.50:0/1258656252 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4003970 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc154d00 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.268+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc1552c0 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.270+0000 7f36c1ffb640 1 -- 172.21.15.50:0/1258656252 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36c401a020 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.270+0000 7f36c1ffb640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36b006c8f0 0x7f36b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.270+0000 7f36c1ffb640 1 -- 172.21.15.50:0/1258656252 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f36c408b340 con 0x7f36cc14e030 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.270+0000 7f36d1417640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36b006c8f0 0x7f36b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.271+0000 7f36d1417640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36b006c8f0 0x7f36b006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f36c8001eb0 tx=0x7f36c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.271+0000 7f36d6264740 1 Processor -- start 2024-04-02T16:23:21.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.271+0000 7f36d6264740 1 -- start start 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9d68b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d98920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d6264740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d69070 0x55f1f9d98e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d6264740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9cd2060 con 0x55f1f9d69070 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9c7a650 con 0x55f1f9d95f70 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f1f9cce010 con 0x55f1f9d95b90 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d69070 0x55f1f9d98e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d69070 0x55f1f9d98e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41064/0 (socket says 172.21.15.50:41064) 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 -- 172.21.15.50:0/2562662325 learned_addr learned my addr 172.21.15.50:0/2562662325 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1417640 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9d68b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d0c16640 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d98920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 -- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 msgr2=0x55f1f9d68b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9d68b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 -- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 msgr2=0x55f1f9d98920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d98920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 -- 172.21.15.50:0/2562662325 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f99e62d0 con 0x55f1f9d69070 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36d1c18640 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d69070 0x55f1f9d98e60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f36c4007fd0 tx=0x7f36c4059730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e605c1066f73dd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36aeffd640 1 -- 172.21.15.50:0/2562662325 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4061b50 con 0x55f1f9d69070 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36aeffd640 1 -- 172.21.15.50:0/2562662325 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c4057010 con 0x55f1f9d69070 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.272+0000 7f36aeffd640 1 -- 172.21.15.50:0/2562662325 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c408b070 con 0x55f1f9d69070 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 -- 172.21.15.50:0/2562662325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d69070 msgr2=0x55f1f9d98e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d69070 0x55f1f9d98e60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f36c4007fd0 tx=0x7f36c4059730 comp rx=0 tx=0).stop 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 -- 172.21.15.50:0/2562662325 shutdown_connections 2024-04-02T16:23:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d69070 0x55f1f9d98e60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d98920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 --2- 172.21.15.50:0/2562662325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9d68b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 -- 172.21.15.50:0/2562662325 >> 172.21.15.50:0/2562662325 conn(0x55f1f9d6ada0 msgr2=0x55f1f9db3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 -- 172.21.15.50:0/2562662325 shutdown_connections 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.273+0000 7f36d6264740 1 -- 172.21.15.50:0/2562662325 wait complete. 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.274+0000 7f36d6264740 1 -- start start 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.274+0000 7f36d6264740 1 Processor -- start 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9db3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9d6d440 con 0x55f1f9d9bef0 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9c8da10 con 0x55f1f9d95f70 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f1f9d6fc80 con 0x55f1f9d95b90 2024-04-02T16:23:21.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 10 client.0 _ll_register_callbacks cb 0x55f1f9cca0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d9c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d9c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41068/0 (socket says 172.21.15.50:41068) 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 -- 172.21.15.50:0/3680476405 learned_addr learned my addr 172.21.15.50:0/3680476405 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d0c16640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1417640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9db3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 -- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 msgr2=0x55f1f9db3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9db3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 -- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 msgr2=0x55f1f9d866f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d866f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 -- 172.21.15.50:0/3680476405 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f9ab06b0 con 0x55f1f9d9bef0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d0c16640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d1c18640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d9c370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f36c40013d0 tx=0x7f36c4001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23889d082dd9240f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36a7fff640 1 -- 172.21.15.50:0/3680476405 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c408b070 con 0x55f1f9d9bef0 2024-04-02T16:23:21.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36a7fff640 1 -- 172.21.15.50:0/3680476405 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f36c40555e0 con 0x55f1f9d9bef0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.275+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c4057010 con 0x55f1f9d9bef0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36a7fff640 1 -- 172.21.15.50:0/3680476405 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f36c40616c0 con 0x55f1f9d9bef0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 10 client.5571 fetch_fsmap learned FSMap version 16 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55f1f9d9a3f0 con 0x55f1f9d9bef0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36a7fff640 1 -- 172.21.15.50:0/3680476405 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c4057010 con 0x55f1f9d9bef0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36a7fff640 1 -- 172.21.15.50:0/3680476405 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f36c40583c0 con 0x55f1f9d9bef0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 10 client.5571 fetch_fsmap finished waiting for FSMap version 16 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 10 client.5571 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 20 client.5571 populate_metadata read hostname 'smithi050' 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9d97b60 0x55f1f9d97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 4 client.5571 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 --> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1f9c79cf0 con 0x55f1f9d97b60 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d1417640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9d97b60 0x55f1f9d97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36a7fff640 10 client.5571 ms_handle_connect on v2:172.21.15.50:6833/3098609271 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.276+0000 7f36d1417640 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9d97b60 0x55f1f9d97f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53fff9b5c4c7ad5f server_cookie=c778a172e37fe157 in_seq=0 out_seq=0 2024-04-02T16:23:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36a7fff640 1 -- 172.21.15.50:0/3680476405 <== mds.0 v2:172.21.15.50:6833/3098609271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1f9c79cf0 con 0x55f1f9d97b60 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36a7fff640 10 client.5571 handle_command_reply: tid=0 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 client.5571 shutdown 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d9bef0 msgr2=0x55f1f9d9c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d9c370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f36c40013d0 tx=0x7f36c4001360 comp rx=0 tx=0).stop 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 shutdown_connections 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9d97b60 0x55f1f9d97f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d9c370 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d95f70 0x55f1f9d866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 --2- 172.21.15.50:0/3680476405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9db3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36a7fff640 0 client.5571 ms_handle_reset on v2:172.21.15.50:6833/3098609271 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 >> 172.21.15.50:0/3680476405 conn(0x55f1f9d69070 msgr2=0x55f1f9d6cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 shutdown_connections 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 1 -- 172.21.15.50:0/3680476405 wait complete. 2024-04-02T16:23:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.277+0000 7f36d6264740 20 client.5571 trim_cache size 0 max 16384 2024-04-02T16:23:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 Processor -- start 2024-04-02T16:23:21.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 -- start start 2024-04-02T16:23:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9d68310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95f70 0x55f1f9d6ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9dd1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1fd380 con 0x55f1f9d95f70 2024-04-02T16:23:21.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc0b5880 con 0x55f1f9d9bef0 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.296+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc0b4be0 con 0x55f1f9d95b90 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95f70 0x55f1f9d6ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95f70 0x55f1f9d6ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41070/0 (socket says 172.21.15.50:41070) 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 -- 172.21.15.50:0/556875224 learned_addr learned my addr 172.21.15.50:0/556875224 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 -- 172.21.15.50:0/556875224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 msgr2=0x55f1f9d68310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 --2- 172.21.15.50:0/556875224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9d68310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 -- 172.21.15.50:0/556875224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 msgr2=0x55f1f9dd1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 --2- 172.21.15.50:0/556875224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9dd1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 -- 172.21.15.50:0/556875224 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f9d69e00 con 0x55f1f9d95f70 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36d0c16640 1 --2- 172.21.15.50:0/556875224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95f70 0x55f1f9d6ca20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009bb0 tx=0x7f36bc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a489dcfc1f5035 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36ae7fc640 1 -- 172.21.15.50:0/556875224 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc043020 con 0x55f1f9d95f70 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36ae7fc640 1 -- 172.21.15.50:0/556875224 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc02c880 con 0x55f1f9d95f70 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.297+0000 7f36ae7fc640 1 -- 172.21.15.50:0/556875224 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc039040 con 0x55f1f9d95f70 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 -- 172.21.15.50:0/556875224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95f70 msgr2=0x55f1f9d6ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 --2- 172.21.15.50:0/556875224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95f70 0x55f1f9d6ca20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009bb0 tx=0x7f36bc02e000 comp rx=0 tx=0).stop 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 -- 172.21.15.50:0/556875224 shutdown_connections 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 --2- 172.21.15.50:0/556875224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9dd1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 --2- 172.21.15.50:0/556875224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95f70 0x55f1f9d6ca20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 --2- 172.21.15.50:0/556875224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95b90 0x55f1f9d68310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 -- 172.21.15.50:0/556875224 >> 172.21.15.50:0/556875224 conn(0x55f1f9d69070 msgr2=0x55f1f9d8cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 -- 172.21.15.50:0/556875224 shutdown_connections 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 -- 172.21.15.50:0/556875224 wait complete. 2024-04-02T16:23:21.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 -- start start 2024-04-02T16:23:21.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.298+0000 7f36d6264740 1 Processor -- start 2024-04-02T16:23:21.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d6264740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95b90 0x55f1f9e30f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95f70 0x55f1f9d8adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d6264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d8b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d6264740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f1f9c8da10 con 0x55f1f9d95b90 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc0b5230 con 0x55f1f9d9bef0 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d6264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f1f9d6d440 con 0x55f1f9d95f70 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d6264740 10 client.0 _ll_register_callbacks cb 0x7f36c40588d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d1417640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95b90 0x55f1f9e30f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d1417640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95b90 0x55f1f9e30f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41074/0 (socket says 172.21.15.50:41074) 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d1417640 1 -- 172.21.15.50:0/3166414799 learned_addr learned my addr 172.21.15.50:0/3166414799 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d1c18640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d8b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.299+0000 7f36d0c16640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95f70 0x55f1f9d8adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d1417640 1 -- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95f70 msgr2=0x55f1f9d8adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d1417640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95f70 0x55f1f9d8adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d1417640 1 -- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 msgr2=0x55f1f9d8b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d1417640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d8b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d1417640 1 -- 172.21.15.50:0/3166414799 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f9d67d60 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d1c18640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d8b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d0c16640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95f70 0x55f1f9d8adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d1417640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95b90 0x55f1f9e30f90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f36c80017e0 tx=0x7f36c8015a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425f86d9119f5068 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36a7fff640 1 -- 172.21.15.50:0/3166414799 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c802d060 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36a7fff640 1 -- 172.21.15.50:0/3166414799 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f36c8025cc0 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36bc02c880 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36a7fff640 1 -- 172.21.15.50:0/3166414799 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f36c8027620 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 10 client.5577 fetch_fsmap learned FSMap version 16 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55f1f9dda850 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36a7fff640 1 -- 172.21.15.50:0/3166414799 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc02c880 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36a7fff640 1 -- 172.21.15.50:0/3166414799 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f36c8021040 con 0x55f1f9d95b90 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 10 client.5577 fetch_fsmap finished waiting for FSMap version 16 2024-04-02T16:23:21.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 10 client.5577 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 20 client.5577 populate_metadata read hostname 'smithi050' 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9dd9710 0x55f1f9dd9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 4 client.5577 mds_command: new command op to 5382 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.300+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 --> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] -- command(tid 0: {"prefix": "status"}) -- 0x55f1f9cdf5a0 con 0x55f1f9dd9710 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d0c16640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9dd9710 0x55f1f9dd9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36a7fff640 10 client.5577 ms_handle_connect on v2:172.21.15.50:6833/3098609271 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d0c16640 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9dd9710 0x55f1f9dd9af0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62077981a95810bb server_cookie=e53e339f1869063e in_seq=0 out_seq=0 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36a7fff640 1 -- 172.21.15.50:0/3166414799 <== mds.0 v2:172.21.15.50:6833/3098609271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f1f9cdf5a0 con 0x55f1f9dd9710 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36a7fff640 10 client.5577 handle_command_reply: tid=0 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 client.5577 shutdown 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95b90 msgr2=0x55f1f9e30f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95b90 0x55f1f9e30f90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f36c80017e0 tx=0x7f36c8015a70 comp rx=0 tx=0).stop 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 shutdown_connections 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1f9d9bef0 0x55f1f9d8b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1f9d95f70 0x55f1f9d8adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:6833/3098609271,v1:172.21.15.50:6835/3098609271] conn(0x55f1f9dd9710 0x55f1f9dd9af0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 --2- 172.21.15.50:0/3166414799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1f9d95b90 0x55f1f9e30f90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36a7fff640 0 client.5577 ms_handle_reset on v2:172.21.15.50:6833/3098609271 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 >> 172.21.15.50:0/3166414799 conn(0x55f1f9d69070 msgr2=0x55f1f9d68100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 shutdown_connections 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 1 -- 172.21.15.50:0/3166414799 wait complete. 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.301+0000 7f36d6264740 20 client.5577 trim_cache size 0 max 16384 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "057e6c06-7c98-48a0-9312-9e11e9abe7fe", 2024-04-02T16:23:21.308 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 5382, 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 4.7187244240000004, 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 16, 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 39, 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 39, 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 12.303001232 2024-04-02T16:23:21.309 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36b006c8f0 msgr2=0x7f36b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36b006c8f0 0x7f36b006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f36c8001eb0 tx=0x7f36c8009040 comp rx=0 tx=0).stop 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36cc14e030 msgr2=0x7f36cc14e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36cc14e030 0x7f36cc14e4b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f36cc06f190 tx=0x7f36c400eb90 comp rx=0 tx=0).stop 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 shutdown_connections 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f36b006c8f0 0x7f36b006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36cc14e030 0x7f36cc14e4b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36cc1e27c0 0x7f36cc13e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 --2- 172.21.15.50:0/1258656252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b8004800 0x7f36cc153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 >> 172.21.15.50:0/1258656252 conn(0x7f36cc072c10 msgr2=0x7f36cc1597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 shutdown_connections 2024-04-02T16:23:21.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.302+0000 7f36d367b640 1 -- 172.21.15.50:0/1258656252 wait complete. 2024-04-02T16:23:21.313 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.414+0000 7f7428049640 1 Processor -- start 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.414+0000 7f7428049640 1 -- start start 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f7428049640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f742015e590 0x7f74200b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f7428049640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f742006bcf0 0x7f742006e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f7428049640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74200724c0 con 0x7f74200b0790 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f7428049640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f742006c4b0 con 0x7f742006bcf0 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f7428049640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f742015e970 con 0x7f742015e590 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f74265e6640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f742006bcf0 0x7f742006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f74265e6640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f742006bcf0 0x7f742006e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41078/0 (socket says 172.21.15.50:41078) 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f74265e6640 1 -- 172.21.15.50:0/2355940015 learned_addr learned my addr 172.21.15.50:0/2355940015 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f7425de5640 1 --2- 172.21.15.50:0/2355940015 >> v2:172.21.15.188:3300/0 conn(0x7f742015e590 0x7f74200b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f74265e6640 1 -- 172.21.15.50:0/2355940015 >> v2:172.21.15.188:3300/0 conn(0x7f742015e590 msgr2=0x7f74200b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f74265e6640 1 --2- 172.21.15.50:0/2355940015 >> v2:172.21.15.188:3300/0 conn(0x7f742015e590 0x7f74200b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f74265e6640 1 -- 172.21.15.50:0/2355940015 >> v1:172.21.15.188:6789/0 conn(0x7f74200b0790 legacy=0x7f74200ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:23:21.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f74265e6640 1 -- 172.21.15.50:0/2355940015 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742006c1d0 con 0x7f742006bcf0 2024-04-02T16:23:21.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.415+0000 7f7425de5640 1 --2- 172.21.15.50:0/2355940015 >> v2:172.21.15.188:3300/0 conn(0x7f742015e590 0x7f74200b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:23:21.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.416+0000 7f74265e6640 1 --2- 172.21.15.50:0/2355940015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742006e970 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f74200b0420 tx=0x7f741802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67851a3d092545c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.416+0000 7f7424de3640 1 -- 172.21.15.50:0/2355940015 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741802c9b0 con 0x7f742006bcf0 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.416+0000 7f7424de3640 1 -- 172.21.15.50:0/2355940015 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741802cb50 con 0x7f742006bcf0 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.416+0000 7f7424de3640 1 -- 172.21.15.50:0/2355940015 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180343f0 con 0x7f742006bcf0 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 -- 172.21.15.50:0/2355940015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 msgr2=0x7f742006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 --2- 172.21.15.50:0/2355940015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742006e970 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f74200b0420 tx=0x7f741802be10 comp rx=0 tx=0).stop 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 -- 172.21.15.50:0/2355940015 shutdown_connections 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 --2- 172.21.15.50:0/2355940015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742006e970 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 --2- 172.21.15.50:0/2355940015 >> v2:172.21.15.188:3300/0 conn(0x7f742015e590 0x7f74200b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 -- 172.21.15.50:0/2355940015 >> 172.21.15.50:0/2355940015 conn(0x7f74200b6540 msgr2=0x7f74200b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 -- 172.21.15.50:0/2355940015 shutdown_connections 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.417+0000 7f7428049640 1 -- 172.21.15.50:0/2355940015 wait complete. 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.418+0000 7f7428049640 1 Processor -- start 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.418+0000 7f7428049640 1 -- start start 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74200b0790 0x7f74200ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f742015e590 0x7f74200cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74200adfc0 con 0x7f742006bcf0 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7420072a40 con 0x7f74200b0790 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f742006eeb0 con 0x7f742015e590 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742014ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41088/0 (socket says 172.21.15.50:41088) 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 -- 172.21.15.50:0/3377163888 learned_addr learned my addr 172.21.15.50:0/3377163888 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f74265e6640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f742015e590 0x7f74200cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f74255e4640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74200b0790 0x7f74200ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 -- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f742015e590 msgr2=0x7f74200cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f742015e590 0x7f74200cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 -- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74200b0790 msgr2=0x7f74200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74200b0790 0x7f74200ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 -- 172.21.15.50:0/3377163888 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74200ce780 con 0x7f742006bcf0 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7425de5640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742014ee90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f741c00c820 tx=0x7f741c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f74167fc640 1 -- 172.21.15.50:0/3377163888 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c015070 con 0x7f742006bcf0 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f74167fc640 1 -- 172.21.15.50:0/3377163888 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c007a30 con 0x7f742006bcf0 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74200cfa10 con 0x7f742006bcf0 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f74167fc640 1 -- 172.21.15.50:0/3377163888 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c007de0 con 0x7f742006bcf0 2024-04-02T16:23:21.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.419+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74200cda40 con 0x7f742006bcf0 2024-04-02T16:23:21.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.421+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74200756f0 con 0x7f742006bcf0 2024-04-02T16:23:21.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.421+0000 7f74167fc640 1 -- 172.21.15.50:0/3377163888 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f741c018070 con 0x7f742006bcf0 2024-04-02T16:23:21.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.421+0000 7f74167fc640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f740406c8f0 0x7f740406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:23:21.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.421+0000 7f74167fc640 1 -- 172.21.15.50:0/3377163888 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f741c08bb50 con 0x7f742006bcf0 2024-04-02T16:23:21.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.421+0000 7f74255e4640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f740406c8f0 0x7f740406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:23:21.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.422+0000 7f74255e4640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f740406c8f0 0x7f740406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f74201560f0 tx=0x7f7410009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:23:21.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.426+0000 7f74167fc640 1 -- 172.21.15.50:0/3377163888 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f741c059e80 con 0x7f742006bcf0 2024-04-02T16:23:21.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.654+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74200b6ac0 con 0x7f742006bcf0 2024-04-02T16:23:21.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.654+0000 7f74167fc640 1 -- 172.21.15.50:0/3377163888 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f741c059780 con 0x7f742006bcf0 2024-04-02T16:23:21.655 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f740406c8f0 msgr2=0x7f740406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f740406c8f0 0x7f740406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f74201560f0 tx=0x7f7410009040 comp rx=0 tx=0).stop 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 msgr2=0x7f742014ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742014ee90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f741c00c820 tx=0x7f741c00cce0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 shutdown_connections 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f742015e590 0x7f74200cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74200b0790 0x7f74200ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f740406c8f0 0x7f740406edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 --2- 172.21.15.50:0/3377163888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f742006bcf0 0x7f742014ee90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 >> 172.21.15.50:0/3377163888 conn(0x7f74200b6540 msgr2=0x7f742015d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 shutdown_connections 2024-04-02T16:23:21.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21.656+0000 7f7428049640 1 -- 172.21.15.50:0/3377163888 wait complete. 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-02T16:23:21.667 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:23:21.668 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:23:21.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:21.668 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:23:21.683 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42648sec preferred_lft 42648sec 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 32:d4:06:8f:be:eb brd ff:ff:ff:ff:ff:ff 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:23:21.684 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::389b:5ff:fe35:52dc/64 scope link 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether a2:a5:44:49:59:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::a0a5:44ff:fe49:5906/64 scope link 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 32:d4:06:8f:be:eb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::30d4:6ff:fe8f:beeb/64 scope link 2024-04-02T16:23:21.685 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:23:21.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:21.686 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:23:21.748 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-04-02T16:23:21.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:21.750 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-04-02T16:23:21.805 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-04-02T16:23:21.805 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:23:21.805 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T16:23:21.805 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T16:23:21.805 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:23:21.806 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:23:21.806 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-04-02T16:23:21.806 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T16:23:21.806 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T16:23:21.806 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T16:23:21.806 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-04-02T16:23:21.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:23:21.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:23:21.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:21.941 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-04-02T16:23:21.958 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-02T16:23:21.958 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T16:23:21.958 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:23:21.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:21.958 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-04-02T16:23:22.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:23:22.025 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T16:23:22.079 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:23:22.079 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:23:22.079 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:23:22.134 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-02T16:23:22.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:22.134 DEBUG:teuthology.orchestra.run.smithi050:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:23:22.218 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:23:22.219 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-02T16:23:22.219 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:23:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:23:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-02T16:23:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:23:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:23:22.227 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:invalid new device string format 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: resolved to: "172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.50:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-02T16:23:22.232 INFO:teuthology.orchestra.run.smithi050.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=057e6c06-7c98-48a0-9312-9e11e9abe7fe" will pass to kernel 2024-04-02T16:23:22.233 INFO:teuthology.orchestra.run.smithi050.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-02T16:23:22.233 INFO:teuthology.orchestra.run.smithi050.stdout: You just mounted a file system that supports labels which does not 2024-04-02T16:23:22.233 INFO:teuthology.orchestra.run.smithi050.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T16:23:22.233 INFO:teuthology.orchestra.run.smithi050.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T16:23:22.233 INFO:teuthology.orchestra.run.smithi050.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T16:23:22.235 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T16:23:22.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:22.235 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:23:22.302 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T16:23:22.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:23:22.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:23:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:23:22.401 INFO:teuthology.orchestra.run.smithi050.stdout:client5586 2024-04-02T16:23:22.401 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:23:22.401 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:23:22.416 DEBUG:teuthology.orchestra.run.smithi050:> mktemp -d --suffix=xfstests-dev 2024-04-02T16:23:22.471 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.CCuraAyba9xfstests-dev 2024-04-02T16:23:22.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:22.471 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.CCuraAyba9xfstests-dev) 2024-04-02T16:23:22.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:22.539 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.CCuraAyba9xfstests-dev) 2024-04-02T16:23:22.595 INFO:teuthology.orchestra.run.smithi050.stderr:Cloning into '/tmp/tmp.CCuraAyba9xfstests-dev'... 2024-04-02T16:23:23.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:23.335 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-02T16:23:23.396 DEBUG:teuthology.orchestra.run.smithi050:> mktemp -d --suffix=test 2024-04-02T16:23:23.451 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.rLt4hGA3Uftest 2024-04-02T16:23:23.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:23:23.451 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-02T16:23:23.507 DEBUG:teuthology.orchestra.run.smithi050:> mktemp -d --suffix=scratch 2024-04-02T16:23:23.562 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:23:23.562 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-04-02T16:23:23.563 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-04-02T16:23:24.182 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:03:49 ago on Tue 02 Apr 2024 04:19:35 PM UTC. 2024-04-02T16:23:24.283 INFO:teuthology.orchestra.run.smithi050.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-04-02T16:23:24.284 INFO:teuthology.orchestra.run.smithi050.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T16:23:24.285 INFO:teuthology.orchestra.run.smithi050.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-04-02T16:23:24.286 INFO:teuthology.orchestra.run.smithi050.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-04-02T16:23:24.287 INFO:teuthology.orchestra.run.smithi050.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-04-02T16:23:24.289 INFO:teuthology.orchestra.run.smithi050.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-04-02T16:23:24.291 INFO:teuthology.orchestra.run.smithi050.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-04-02T16:23:24.293 INFO:teuthology.orchestra.run.smithi050.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-04-02T16:23:24.295 INFO:teuthology.orchestra.run.smithi050.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2024-04-02T16:23:24.298 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-02T16:23:24.299 INFO:teuthology.orchestra.run.smithi050.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-04-02T16:23:24.301 INFO:teuthology.orchestra.run.smithi050.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-04-02T16:23:24.302 INFO:teuthology.orchestra.run.smithi050.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-04-02T16:23:24.303 INFO:teuthology.orchestra.run.smithi050.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-04-02T16:23:24.304 INFO:teuthology.orchestra.run.smithi050.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-04-02T16:23:24.306 INFO:teuthology.orchestra.run.smithi050.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-04-02T16:23:24.307 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-04-02T16:23:24.308 INFO:teuthology.orchestra.run.smithi050.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-04-02T16:23:24.309 INFO:teuthology.orchestra.run.smithi050.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-04-02T16:23:24.312 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-02T16:23:24.381 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-04-02T16:23:24.385 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-devel x86_64 5.14.0-434.el9 CentOS-AppStream 16 M 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: libcap-devel x86_64 2.48-9.el9 CentOS-AppStream 33 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: sqlite x86_64 3.34.1-7.el9 CentOS-AppStream 747 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: udftools x86_64 2.2-5.el9 CentOS-AppStream 144 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: cpp x86_64 11.4.1-3.el9 appstream 11 M 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper x86_64 9:1.02.197-2.el9 baseos 141 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper-event x86_64 9:1.02.197-2.el9 baseos 34 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper-event-libs x86_64 9:1.02.197-2.el9 baseos 32 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper-libs x86_64 9:1.02.197-2.el9 baseos 179 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: e2fsprogs x86_64 1.46.5-5.el9 baseos 1.0 M 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: e2fsprogs-libs x86_64 1.46.5-5.el9 baseos 222 k 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: gcc x86_64 11.4.1-3.el9 appstream 32 M 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: gcc-c++ x86_64 11.4.1-3.el9 appstream 13 M 2024-04-02T16:23:24.386 INFO:teuthology.orchestra.run.smithi050.stdout: gcc-plugin-annobin x86_64 11.4.1-3.el9 appstream 47 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: git x86_64 2.43.0-1.el9 appstream 51 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: git-core x86_64 2.43.0-1.el9 appstream 4.4 M 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: git-core-doc noarch 2.43.0-1.el9 appstream 2.9 M 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: libcom_err x86_64 1.46.5-5.el9 baseos 26 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: libgcc x86_64 11.4.1-3.el9 baseos 92 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: libgomp x86_64 11.4.1-3.el9 baseos 268 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: libss x86_64 1.46.5-5.el9 baseos 32 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: libstdc++ x86_64 11.4.1-3.el9 baseos 743 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: libstdc++-devel x86_64 11.4.1-3.el9 appstream 2.4 M 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: lvm2 x86_64 9:2.03.23-2.el9 baseos 1.5 M 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: lvm2-libs x86_64 9:2.03.23-2.el9 baseos 1.0 M 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: make x86_64 1:4.3-8.el9 baseos 536 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: perl-Git noarch 2.43.0-1.el9 appstream 38 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: python-unversioned-command noarch 3.9.18-3.el9 CentOS-AppStream 8.3 k 2024-04-02T16:23:24.387 INFO:teuthology.orchestra.run.smithi050.stdout: python3 x86_64 3.9.18-3.el9 CentOS-BaseOS 25 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.18-3.el9 CentOS-AppStream 204 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: python3-libs x86_64 3.9.18-3.el9 CentOS-BaseOS 7.3 M 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: sqlite-libs x86_64 3.34.1-7.el9 CentOS-BaseOS 619 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:24.388 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-04-02T16:23:24.389 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T16:23:24.389 INFO:teuthology.orchestra.run.smithi050.stdout:Install 17 Packages 2024-04-02T16:23:24.389 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 28 Packages 2024-04-02T16:23:24.389 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:23:24.389 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 103 M 2024-04-02T16:23:24.389 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-04-02T16:23:25.097 INFO:teuthology.orchestra.run.smithi050.stdout:(1/45): fio-engine-http-3.35-1.el9.x86_64.rpm 102 kB/s | 17 kB 00:00 2024-04-02T16:23:25.130 INFO:teuthology.orchestra.run.smithi050.stdout:(2/45): fio-engine-libaio-3.35-1.el9.x86_64.rpm 72 kB/s | 14 kB 00:00 2024-04-02T16:23:25.555 INFO:teuthology.orchestra.run.smithi050.stdout:(3/45): fio-engine-nbd-3.35-1.el9.x86_64.rpm 29 kB/s | 13 kB 00:00 2024-04-02T16:23:25.581 INFO:teuthology.orchestra.run.smithi050.stdout:(4/45): fio-engine-rados-3.35-1.el9.x86_64.rpm 32 kB/s | 15 kB 00:00 2024-04-02T16:23:25.781 INFO:teuthology.orchestra.run.smithi050.stdout:(5/45): fio-engine-rbd-3.35-1.el9.x86_64.rpm 67 kB/s | 15 kB 00:00 2024-04-02T16:23:25.798 INFO:teuthology.orchestra.run.smithi050.stdout:(6/45): fio-engine-rdma-3.35-1.el9.x86_64.rpm 90 kB/s | 20 kB 00:00 2024-04-02T16:23:25.965 INFO:teuthology.orchestra.run.smithi050.stdout:(7/45): libcap-devel-2.48-9.el9.x86_64.rpm 198 kB/s | 33 kB 00:00 2024-04-02T16:23:26.315 INFO:teuthology.orchestra.run.smithi050.stdout:(8/45): fio-3.35-1.el9.x86_64.rpm 4.5 MB/s | 6.2 MB 00:01 2024-04-02T16:23:26.399 INFO:teuthology.orchestra.run.smithi050.stdout:(9/45): sqlite-3.34.1-7.el9.x86_64.rpm 1.7 MB/s | 747 kB 00:00 2024-04-02T16:23:26.532 INFO:teuthology.orchestra.run.smithi050.stdout:(10/45): udftools-2.2-5.el9.x86_64.rpm 663 kB/s | 144 kB 00:00 2024-04-02T16:23:26.591 INFO:teuthology.orchestra.run.smithi050.stdout:(11/45): rmt-1.6-6.el9.x86_64.rpm 236 kB/s | 45 kB 00:00 2024-04-02T16:23:26.883 INFO:teuthology.orchestra.run.smithi050.stdout:(12/45): gdbm-1.19-4.el9.x86_64.rpm 454 kB/s | 132 kB 00:00 2024-04-02T16:23:26.967 INFO:teuthology.orchestra.run.smithi050.stdout:(13/45): gdbm-devel-1.19-4.el9.x86_64.rpm 718 kB/s | 59 kB 00:00 2024-04-02T16:23:27.134 INFO:teuthology.orchestra.run.smithi050.stdout:(14/45): kernel-devel-5.14.0-434.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-02T16:23:27.159 INFO:teuthology.orchestra.run.smithi050.stdout:(15/45): libnbd-1.18.1-3.el9.x86_64.rpm 266 kB/s | 166 kB 00:00 2024-04-02T16:23:27.193 INFO:teuthology.orchestra.run.smithi050.stdout:(16/45): python-unversioned-command-3.9.18-3.el 249 kB/s | 8.3 kB 00:00 2024-04-02T16:23:27.234 INFO:teuthology.orchestra.run.smithi050.stdout:(17/45): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 204 kB 00:00 2024-04-02T16:23:27.268 INFO:teuthology.orchestra.run.smithi050.stdout:(18/45): python3-3.9.18-3.el9.x86_64.rpm 757 kB/s | 25 kB 00:00 2024-04-02T16:23:27.619 INFO:teuthology.orchestra.run.smithi050.stdout:(19/45): python3-libs-3.9.18-3.el9.x86_64.rpm 21 MB/s | 7.3 MB 00:00 2024-04-02T16:23:27.644 INFO:teuthology.orchestra.run.smithi050.stdout:(20/45): dump-0.4-0.51.b47.el9.x86_64.rpm 130 kB/s | 88 kB 00:00 2024-04-02T16:23:27.694 INFO:teuthology.orchestra.run.smithi050.stdout:(21/45): indent-2.2.13-5.el9.x86_64.rpm 321 kB/s | 179 kB 00:00 2024-04-02T16:23:27.720 INFO:teuthology.orchestra.run.smithi050.stdout:(22/45): sqlite-libs-3.34.1-7.el9.x86_64.rpm 6.0 MB/s | 619 kB 00:00 2024-04-02T16:23:27.753 INFO:teuthology.orchestra.run.smithi050.stdout:(23/45): device-mapper-1.02.197-2.el9.x86_64.rp 1.3 MB/s | 141 kB 00:00 2024-04-02T16:23:27.787 INFO:teuthology.orchestra.run.smithi050.stdout:(24/45): device-mapper-event-1.02.197-2.el9.x86 373 kB/s | 34 kB 00:00 2024-04-02T16:23:27.821 INFO:teuthology.orchestra.run.smithi050.stdout:(25/45): device-mapper-event-libs-1.02.197-2.el 321 kB/s | 32 kB 00:00 2024-04-02T16:23:27.888 INFO:teuthology.orchestra.run.smithi050.stdout:(26/45): device-mapper-libs-1.02.197-2.el9.x86_ 1.3 MB/s | 179 kB 00:00 2024-04-02T16:23:28.055 INFO:teuthology.orchestra.run.smithi050.stdout:(27/45): libcom_err-1.46.5-5.el9.x86_64.rpm 158 kB/s | 26 kB 00:00 2024-04-02T16:23:28.088 INFO:teuthology.orchestra.run.smithi050.stdout:(28/45): e2fsprogs-1.46.5-5.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-04-02T16:23:28.114 INFO:teuthology.orchestra.run.smithi050.stdout:(29/45): e2fsprogs-libs-1.46.5-5.el9.x86_64.rpm 759 kB/s | 222 kB 00:00 2024-04-02T16:23:28.139 INFO:teuthology.orchestra.run.smithi050.stdout:(30/45): libgcc-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 92 kB 00:00 2024-04-02T16:23:28.222 INFO:teuthology.orchestra.run.smithi050.stdout:(31/45): libgomp-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 268 kB 00:00 2024-04-02T16:23:28.248 INFO:teuthology.orchestra.run.smithi050.stdout:(32/45): libss-1.46.5-5.el9.x86_64.rpm 238 kB/s | 32 kB 00:00 2024-04-02T16:23:28.298 INFO:teuthology.orchestra.run.smithi050.stdout:(33/45): libstdc++-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 743 kB 00:00 2024-04-02T16:23:28.340 INFO:teuthology.orchestra.run.smithi050.stdout:(34/45): lvm2-2.03.23-2.el9.x86_64.rpm 13 MB/s | 1.5 MB 00:00 2024-04-02T16:23:28.373 INFO:teuthology.orchestra.run.smithi050.stdout:(35/45): make-4.3-8.el9.x86_64.rpm 7.0 MB/s | 536 kB 00:00 2024-04-02T16:23:28.407 INFO:teuthology.orchestra.run.smithi050.stdout:(36/45): lvm2-libs-2.03.23-2.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-04-02T16:23:32.522 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:23:32.526 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:23:35.507 INFO:teuthology.orchestra.run.smithi050.stdout:(37/45): gcc-c++-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 13 MB 00:07 2024-04-02T16:23:35.608 INFO:teuthology.orchestra.run.smithi050.stdout:(38/45): gcc-plugin-annobin-11.4.1-3.el9.x86_64 473 kB/s | 47 kB 00:00 2024-04-02T16:23:35.700 INFO:teuthology.orchestra.run.smithi050.stdout:(39/45): git-2.43.0-1.el9.x86_64.rpm 552 kB/s | 51 kB 00:00 2024-04-02T16:23:36.167 INFO:teuthology.orchestra.run.smithi050.stdout:(40/45): git-core-2.43.0-1.el9.x86_64.rpm 9.5 MB/s | 4.4 MB 00:00 2024-04-02T16:23:36.450 INFO:teuthology.orchestra.run.smithi050.stdout:(41/45): git-core-doc-2.43.0-1.el9.noarch.rpm 10 MB/s | 2.9 MB 00:00 2024-04-02T16:23:36.684 INFO:teuthology.orchestra.run.smithi050.stdout:(42/45): libstdc++-devel-11.4.1-3.el9.x86_64.rp 10 MB/s | 2.4 MB 00:00 2024-04-02T16:23:36.784 INFO:teuthology.orchestra.run.smithi050.stdout:(43/45): perl-Git-2.43.0-1.el9.noarch.rpm 378 kB/s | 38 kB 00:00 2024-04-02T16:23:37.702 INFO:teuthology.orchestra.run.smithi050.stdout:(44/45): cpp-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 11 MB 00:09 2024-04-02T16:23:54.185 INFO:teuthology.orchestra.run.smithi050.stdout:(45/45): gcc-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 32 MB 00:25 2024-04-02T16:23:54.185 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-04-02T16:23:54.186 INFO:teuthology.orchestra.run.smithi050.stdout:Total 3.5 MB/s | 103 MB 00:29 2024-04-02T16:23:55.024 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-04-02T16:23:55.344 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-04-02T16:23:55.344 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-04-02T16:23:56.684 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-04-02T16:23:56.685 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-04-02T16:23:59.724 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-04-02T16:23:59.790 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libgcc-11.4.1-3.el9.x86_64 1/73 2024-04-02T16:23:59.853 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libgcc-11.4.1-3.el9.x86_64 1/73 2024-04-02T16:23:59.925 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libcom_err-1.46.5-5.el9.x86_64 2/73 2024-04-02T16:23:59.982 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libstdc++-11.4.1-3.el9.x86_64 3/73 2024-04-02T16:24:00.030 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : e2fsprogs-libs-1.46.5-5.el9.x86_64 4/73 2024-04-02T16:24:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : device-mapper-libs-9:1.02.197-2.el9.x86_64 5/73 2024-04-02T16:24:00.131 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : device-mapper-9:1.02.197-2.el9.x86_64 6/73 2024-04-02T16:24:00.345 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 7/73 2024-04-02T16:24:00.544 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : git-core-2.43.0-1.el9.x86_64 8/73 2024-04-02T16:24:00.612 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : make-1:4.3-8.el9.x86_64 9/73 2024-04-02T16:24:01.951 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : sqlite-libs-3.34.1-7.el9.x86_64 10/73 2024-04-02T16:24:02.031 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : python3-libs-3.9.18-3.el9.x86_64 11/73 2024-04-02T16:24:02.070 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : python3-3.9.18-3.el9.x86_64 12/73 2024-04-02T16:24:02.659 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:24:02.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:24:02.949 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : python-unversioned-command-3.9.18-3.el9.noarch 13/73 2024-04-02T16:24:03.009 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : git-core-doc-2.43.0-1.el9.noarch 14/73 2024-04-02T16:24:03.050 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : perl-Git-2.43.0-1.el9.noarch 15/73 2024-04-02T16:24:03.094 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : git-2.43.0-1.el9.x86_64 16/73 2024-04-02T16:24:03.126 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-04-02T16:24:03.354 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-04-02T16:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : lvm2-libs-9:2.03.23-2.el9.x86_64 18/73 2024-04-02T16:24:04.033 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libstdc++-devel-11.4.1-3.el9.x86_64 19/73 2024-04-02T16:24:04.399 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libss-1.46.5-5.el9.x86_64 20/73 2024-04-02T16:24:04.458 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : cpp-11.4.1-3.el9.x86_64 21/73 2024-04-02T16:24:05.586 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libgomp-11.4.1-3.el9.x86_64 22/73 2024-04-02T16:24:05.812 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : gcc-11.4.1-3.el9.x86_64 23/73 2024-04-02T16:24:05.855 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: gcc-11.4.1-3.el9.x86_64 23/73 2024-04-02T16:24:05.895 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 24/73 2024-04-02T16:24:05.924 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 25/73 2024-04-02T16:24:05.948 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 26/73 2024-04-02T16:24:05.974 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 27/73 2024-04-02T16:24:06.008 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 28/73 2024-04-02T16:24:06.033 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 29/73 2024-04-02T16:24:06.058 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 30/73 2024-04-02T16:24:06.225 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 31/73 2024-04-02T16:24:06.264 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fio-3.35-1.el9.x86_64 32/73 2024-04-02T16:24:06.300 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rmt-2:1.6-6.el9.x86_64 33/73 2024-04-02T16:24:06.330 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 34/73 2024-04-02T16:24:16.260 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 35/73 2024-04-02T16:24:16.462 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-devel-5.14.0-434.el9.x86_64 36/73 2024-04-02T16:24:26.698 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-devel-5.14.0-434.el9.x86_64 36/73 2024-04-02T16:24:26.755 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : gcc-c++-11.4.1-3.el9.x86_64 37/73 2024-04-02T16:24:26.803 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-04-02T16:24:26.915 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-04-02T16:24:27.109 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : e2fsprogs-1.46.5-5.el9.x86_64 39/73 2024-04-02T16:24:27.109 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-04-02T16:24:27.109 INFO:teuthology.orchestra.run.smithi050.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2024-04-02T16:24:27.110 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:24:27.407 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-04-02T16:24:27.462 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : python3-devel-3.9.18-3.el9.x86_64 41/73 2024-04-02T16:24:27.512 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 42/73 2024-04-02T16:24:27.558 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : indent-2.2.13-5.el9.x86_64 43/73 2024-04-02T16:24:27.608 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : udftools-2.2-5.el9.x86_64 44/73 2024-04-02T16:24:27.633 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 45/73 2024-04-02T16:24:27.661 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : git-2.39.3-1.el9.x86_64 46/73 2024-04-02T16:24:27.694 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 47/73 2024-04-02T16:24:27.727 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 48/73 2024-04-02T16:24:27.752 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 49/73 2024-04-02T16:24:27.777 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 50/73 2024-04-02T16:24:27.794 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 51/73 2024-04-02T16:24:27.795 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-04-02T16:24:27.833 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-04-02T16:24:27.863 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 53/73 2024-04-02T16:24:27.870 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-04-02T16:24:27.871 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-04-02T16:24:27.925 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-04-02T16:24:27.980 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 55/73 2024-04-02T16:24:28.014 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : git-core-doc-2.39.3-1.el9.noarch 56/73 2024-04-02T16:24:28.039 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : perl-Git-2.39.3-1.el9.noarch 57/73 2024-04-02T16:24:28.064 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : lvm2-libs-9:2.03.21-3.el9.x86_64 58/73 2024-04-02T16:24:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-04-02T16:24:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-04-02T16:24:28.124 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 60/73 2024-04-02T16:24:28.149 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 61/73 2024-04-02T16:24:28.174 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 62/73 2024-04-02T16:24:28.200 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 63/73 2024-04-02T16:24:28.225 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : device-mapper-libs-9:1.02.195-3.el9.x86_64 64/73 2024-04-02T16:24:28.288 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : device-mapper-9:1.02.195-3.el9.x86_64 65/73 2024-04-02T16:24:28.346 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 66/73 2024-04-02T16:24:28.370 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : sqlite-libs-3.34.1-6.el9.x86_64 67/73 2024-04-02T16:24:28.396 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 68/73 2024-04-02T16:24:28.396 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-04-02T16:24:28.484 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-04-02T16:24:28.524 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : git-core-2.39.3-1.el9.x86_64 70/73 2024-04-02T16:24:28.549 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 71/73 2024-04-02T16:24:28.575 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 72/73 2024-04-02T16:24:28.646 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : make-1:4.3-7.el9.x86_64 73/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: make-1:4.3-7.el9.x86_64 73/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fio-3.35-1.el9.x86_64 1/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/73 2024-04-02T16:24:31.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-devel-5.14.0-434.el9.x86_64 8/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 9/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 10/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : udftools-2.2-5.el9.x86_64 11/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 12/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 13/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/73 2024-04-02T16:24:31.029 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-unversioned-command-3.9.18-3.el9.noarch 18/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 19/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 20/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 21/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-3.9.18-3.el9.x86_64 22/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-3.9.17-2.el9.x86_64 23/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-libs-3.9.18-3.el9.x86_64 24/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 25/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : sqlite-libs-3.34.1-7.el9.x86_64 26/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : sqlite-libs-3.34.1-6.el9.x86_64 27/73 2024-04-02T16:24:31.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-9:1.02.197-2.el9.x86_64 28/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-9:1.02.195-3.el9.x86_64 29/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 30/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-event-9:1.02.195-3.el9.x86_64 31/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 32/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 33/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-libs-9:1.02.197-2.el9.x86_64 34/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : device-mapper-libs-9:1.02.195-3.el9.x86_64 35/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : e2fsprogs-1.46.5-5.el9.x86_64 36/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 37/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : e2fsprogs-libs-1.46.5-5.el9.x86_64 38/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 39/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcom_err-1.46.5-5.el9.x86_64 40/73 2024-04-02T16:24:31.035 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 41/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgcc-11.4.1-3.el9.x86_64 42/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 43/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgomp-11.4.1-3.el9.x86_64 44/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 45/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libss-1.46.5-5.el9.x86_64 46/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libss-1.46.5-3.el9.x86_64 47/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libstdc++-11.4.1-3.el9.x86_64 48/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 49/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lvm2-9:2.03.23-2.el9.x86_64 50/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lvm2-9:2.03.21-3.el9.x86_64 51/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 52/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lvm2-libs-9:2.03.21-3.el9.x86_64 53/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : make-1:4.3-8.el9.x86_64 54/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : make-1:4.3-7.el9.x86_64 55/73 2024-04-02T16:24:31.036 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cpp-11.4.1-3.el9.x86_64 56/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 57/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gcc-11.4.1-3.el9.x86_64 58/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 59/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gcc-c++-11.4.1-3.el9.x86_64 60/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 61/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gcc-plugin-annobin-11.4.1-3.el9.x86_64 62/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 63/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : git-2.43.0-1.el9.x86_64 64/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : git-2.39.3-1.el9.x86_64 65/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : git-core-2.43.0-1.el9.x86_64 66/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : git-core-2.39.3-1.el9.x86_64 67/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : git-core-doc-2.43.0-1.el9.noarch 68/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : git-core-doc-2.39.3-1.el9.noarch 69/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libstdc++-devel-11.4.1-3.el9.x86_64 70/73 2024-04-02T16:24:31.037 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 71/73 2024-04-02T16:24:31.038 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : perl-Git-2.43.0-1.el9.noarch 72/73 2024-04-02T16:24:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : perl-Git-2.39.3-1.el9.noarch 73/73 2024-04-02T16:24:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:24:32.038 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-04-02T16:24:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: cpp-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper-9:1.02.197-2.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper-event-9:1.02.197-2.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper-event-libs-9:1.02.197-2.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: device-mapper-libs-9:1.02.197-2.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: e2fsprogs-1.46.5-5.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: e2fsprogs-libs-1.46.5-5.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: gcc-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: gcc-c++-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: gcc-plugin-annobin-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: git-2.43.0-1.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: git-core-2.43.0-1.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: git-core-doc-2.43.0-1.el9.noarch 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: libcom_err-1.46.5-5.el9.x86_64 2024-04-02T16:24:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: libgcc-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: libgomp-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: libss-1.46.5-5.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: libstdc++-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: libstdc++-devel-11.4.1-3.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: lvm2-9:2.03.23-2.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: lvm2-libs-9:2.03.23-2.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: make-1:4.3-8.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: perl-Git-2.43.0-1.el9.noarch 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: python-unversioned-command-3.9.18-3.el9.noarch 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: python3-3.9.18-3.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: python3-libs-3.9.18-3.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: sqlite-libs-3.34.1-7.el9.x86_64 2024-04-02T16:24:32.040 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-434.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: udftools-2.2-5.el9.x86_64 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:24:32.041 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-04-02T16:24:32.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo useradd -m fsgqa 2024-04-02T16:24:32.779 INFO:teuthology.orchestra.run.smithi050.stderr:useradd: user 'fsgqa' already exists 2024-04-02T16:24:32.780 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-02T16:24:32.781 DEBUG:teuthology.orchestra.run.smithi050:> sudo groupadd fsgqa 2024-04-02T16:24:32.783 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:24:32.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:24:32.849 INFO:teuthology.orchestra.run.smithi050.stderr:groupadd: group 'fsgqa' already exists 2024-04-02T16:24:32.850 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-02T16:24:32.851 DEBUG:teuthology.orchestra.run.smithi050:> sudo useradd fsgqa2 2024-04-02T16:24:33.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo useradd 123456-fsgqa 2024-04-02T16:24:34.415 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-02T16:24:34.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.517+0000 7f547f688640 1 Processor -- start 2024-04-02T16:24:34.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.517+0000 7f547f688640 1 -- start start 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.517+0000 7f547f688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54780cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.517+0000 7f547f688640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f547813e1d0 0x7f54780cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.517+0000 7f547f688640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f54780ccde0 0x7f547815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547f688640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f54780ce820 con 0x7f54780ccde0 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547f688640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f54780ceac0 con 0x7f547813e1d0 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547f688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f54780ceda0 con 0x7f547813ddf0 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f54780ccde0 0x7f547815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f54780ccde0 0x7f547815ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45040/0 (socket says 172.21.15.50:45040) 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 -- 172.21.15.50:0/3501151756 learned_addr learned my addr 172.21.15.50:0/3501151756 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547cc23640 1 --2- 172.21.15.50:0/3501151756 >> v2:172.21.15.188:3300/0 conn(0x7f547813e1d0 0x7f54780cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547d424640 1 --2- 172.21.15.50:0/3501151756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54780cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 -- 172.21.15.50:0/3501151756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 msgr2=0x7f54780cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 --2- 172.21.15.50:0/3501151756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54780cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 -- 172.21.15.50:0/3501151756 >> v2:172.21.15.188:3300/0 conn(0x7f547813e1d0 msgr2=0x7f54780cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 --2- 172.21.15.50:0/3501151756 >> v2:172.21.15.188:3300/0 conn(0x7f547813e1d0 0x7f54780cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547dc25640 1 -- 172.21.15.50:0/3501151756 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54780cf080 con 0x7f54780ccde0 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.518+0000 7f547d424640 1 --2- 172.21.15.50:0/3501151756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54780cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547dc25640 1 --2- 172.21.15.50:0/3501151756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54780ccde0 0x7f547815ba20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f547806f660 tx=0x7f547002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7ad13fc0960261 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f546ffff640 1 -- 172.21.15.50:0/3501151756 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547002ca70 con 0x7f54780ccde0 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f546ffff640 1 -- 172.21.15.50:0/3501151756 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547002cc10 con 0x7f54780ccde0 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f546ffff640 1 -- 172.21.15.50:0/3501151756 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54700343f0 con 0x7f54780ccde0 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547f688640 1 -- 172.21.15.50:0/3501151756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54780ccde0 msgr2=0x7f547815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547f688640 1 --2- 172.21.15.50:0/3501151756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54780ccde0 0x7f547815ba20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f547806f660 tx=0x7f547002bed0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547f688640 1 -- 172.21.15.50:0/3501151756 shutdown_connections 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547f688640 1 --2- 172.21.15.50:0/3501151756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54780ccde0 0x7f547815ba20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547f688640 1 --2- 172.21.15.50:0/3501151756 >> v2:172.21.15.188:3300/0 conn(0x7f547813e1d0 0x7f54780cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547f688640 1 --2- 172.21.15.50:0/3501151756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54780cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.519+0000 7f547f688640 1 -- 172.21.15.50:0/3501151756 >> 172.21.15.50:0/3501151756 conn(0x7f547806ec30 msgr2=0x7f547806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:24:34.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.520+0000 7f547f688640 1 -- 172.21.15.50:0/3501151756 shutdown_connections 2024-04-02T16:24:34.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.520+0000 7f547f688640 1 -- 172.21.15.50:0/3501151756 wait complete. 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.521+0000 7f547f688640 1 Processor -- start 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.521+0000 7f547f688640 1 -- start start 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547f688640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54780ccde0 0x7f54781573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547f688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54781f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547f688640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f547813e1d0 0x7f54781f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547f688640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f547815c0e0 con 0x7f547813e1d0 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547f688640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f547815bf60 con 0x7f54780ccde0 2024-04-02T16:24:34.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547f688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f547815c260 con 0x7f547813ddf0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f547813e1d0 0x7f54781f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f547813e1d0 0x7f54781f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45046/0 (socket says 172.21.15.50:45046) 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 -- 172.21.15.50:0/4175665039 learned_addr learned my addr 172.21.15.50:0/4175665039 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547d424640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54780ccde0 0x7f54781573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547cc23640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54781f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 -- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 msgr2=0x7f54781f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54781f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 -- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54780ccde0 msgr2=0x7f54781573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54780ccde0 0x7f54781573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 -- 172.21.15.50:0/4175665039 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54781f4e60 con 0x7f547813e1d0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547dc25640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f547813e1d0 0x7f54781f4540 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5470036040 tx=0x7f5470004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f546dffb640 1 -- 172.21.15.50:0/4175665039 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470043020 con 0x7f547813e1d0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f546dffb640 1 -- 172.21.15.50:0/4175665039 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547003e450 con 0x7f547813e1d0 2024-04-02T16:24:34.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54780b1760 con 0x7f547813e1d0 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.522+0000 7f546dffb640 1 -- 172.21.15.50:0/4175665039 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470009510 con 0x7f547813e1d0 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.523+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54780b1d20 con 0x7f547813e1d0 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.523+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54780756f0 con 0x7f547813e1d0 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.524+0000 7f546dffb640 1 -- 172.21.15.50:0/4175665039 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5470046020 con 0x7f547813e1d0 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.524+0000 7f546dffb640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f545c06c8f0 0x7f545c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.524+0000 7f546dffb640 1 -- 172.21.15.50:0/4175665039 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f54700ae580 con 0x7f547813e1d0 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.525+0000 7f547d424640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f545c06c8f0 0x7f545c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.525+0000 7f547d424640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f545c06c8f0 0x7f545c06edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5474006b10 tx=0x7f5474008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:24:34.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.529+0000 7f546dffb640 1 -- 172.21.15.50:0/4175665039 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f547004d030 con 0x7f547813e1d0 2024-04-02T16:24:34.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.785+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f54780b5070 con 0x7f547813e1d0 2024-04-02T16:24:34.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.785+0000 7f546dffb640 1 -- 172.21.15.50:0/4175665039 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7f54781f4e60 con 0x7f547813e1d0 2024-04-02T16:24:34.785 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:24:34.785 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":1,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","modified":"2024-04-02T16:21:03.983028Z","created":"2024-04-02T16:21:03.983028Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3301","nonce":0},{"type":"v1","addr":"172.21.15.188:6790","nonce":0}]},"addr":"172.21.15.188:6790/0","public_addr":"172.21.15.188:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-02T16:24:34.786 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 1 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f545c06c8f0 msgr2=0x7f545c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f545c06c8f0 0x7f545c06edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5474006b10 tx=0x7f5474008040 comp rx=0 tx=0).stop 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f547813e1d0 msgr2=0x7f54781f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f547813e1d0 0x7f54781f4540 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5470036040 tx=0x7f5470004a40 comp rx=0 tx=0).stop 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 shutdown_connections 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f545c06c8f0 0x7f545c06edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f547813e1d0 0x7f54781f4540 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547813ddf0 0x7f54781f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 --2- 172.21.15.50:0/4175665039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54780ccde0 0x7f54781573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 >> 172.21.15.50:0/4175665039 conn(0x7f547806ec30 msgr2=0x7f54780b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 shutdown_connections 2024-04-02T16:24:34.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.787+0000 7f547f688640 1 -- 172.21.15.50:0/4175665039 wait complete. 2024-04-02T16:24:34.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2024-04-02T16:24:34.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.903+0000 7fcf65688640 1 Processor -- start 2024-04-02T16:24:34.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.903+0000 7fcf65688640 1 -- start start 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf65688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 0x7fcf6009aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf65688640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf6009af80 0x7fcf600cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf65688640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6006cc40 con 0x7fcf600cd9d0 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf65688640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6009b3c0 con 0x7fcf6009af80 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf65688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6006c960 con 0x7fcf60071020 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5f7fe640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7fcf600cd9d0 0x7fcf600cde60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:44286/0 (socket says 172.21.15.50:44286) 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5f7fe640 1 -- 172.21.15.50:0/1827620731 learned_addr learned my addr 172.21.15.50:0/1827620731 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5e7fc640 1 --2- 172.21.15.50:0/1827620731 >> v2:172.21.15.188:3300/0 conn(0x7fcf6009af80 0x7fcf600cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 0x7fcf6009aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 <== mon.0 v1:172.21.15.50:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138401160 0 0) 0x7fcf6006cc40 con 0x7fcf600cd9d0 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 --> v1:172.21.15.50:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf4c003520 con 0x7fcf600cd9d0 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 <== mon.0 v1:172.21.15.50:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1201359719 0 0) 0x7fcf4c003520 con 0x7fcf600cd9d0 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 msgr2=0x7fcf6009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:24:34.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 0x7fcf6009aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 >> v2:172.21.15.188:3300/0 conn(0x7fcf6009af80 msgr2=0x7fcf600cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 --2- 172.21.15.50:0/1827620731 >> v2:172.21.15.188:3300/0 conn(0x7fcf6009af80 0x7fcf600cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.904+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 --> v1:172.21.15.50:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf600cc6d0 con 0x7fcf600cd9d0 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 <== mon.0 v1:172.21.15.50:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 819069968 0 0) 0x7fcf500019f0 con 0x7fcf600cd9d0 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 0x7fcf6009aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5e7fc640 1 --2- 172.21.15.50:0/1827620731 >> v2:172.21.15.188:3300/0 conn(0x7fcf6009af80 0x7fcf600cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 >> v1:172.21.15.50:6789/0 conn(0x7fcf600cd9d0 legacy=0x7fcf600cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf4c0039b0 0x7fcf4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c004800 0x7fcf4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 0x7fcf4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6009b3c0 con 0x7fcf4c0039b0 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6006c960 con 0x7fcf4c004800 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c00b9c0 con 0x7fcf60071020 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf4c0039b0 0x7fcf4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5e7fc640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c004800 0x7fcf4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5f7fe640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 0x7fcf4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5effd640 1 -- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 msgr2=0x7fcf4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf60071020 0x7fcf4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5effd640 1 -- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c004800 msgr2=0x7fcf4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c004800 0x7fcf4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5effd640 1 -- 172.21.15.50:0/1827620731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c003520 con 0x7fcf4c0039b0 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.905+0000 7fcf5e7fc640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c004800 0x7fcf4c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf4c0039b0 0x7fcf4c003dd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009c90 tx=0x7fcf4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9798bdee7b5a597c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58039070 con 0x7fcf4c0039b0 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5802c8f0 con 0x7fcf4c0039b0 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf5dffb640 1 -- 172.21.15.50:0/1827620731 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009860 con 0x7fcf4c0039b0 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf65688640 1 -- 172.21.15.50:0/1827620731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf4c0039b0 msgr2=0x7fcf4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf65688640 1 --2- 172.21.15.50:0/1827620731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf4c0039b0 0x7fcf4c003dd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009c90 tx=0x7fcf4c005830 comp rx=0 tx=0).stop 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf5f7fe640 1 -- 172.21.15.50:0/1827620731 reap_dead start 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf65688640 1 -- 172.21.15.50:0/1827620731 shutdown_connections 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf65688640 1 -- 172.21.15.50:0/1827620731 >> 172.21.15.50:0/1827620731 conn(0x7fcf600b3ad0 msgr2=0x7fcf600b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:24:34.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf65688640 1 -- 172.21.15.50:0/1827620731 shutdown_connections 2024-04-02T16:24:34.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.906+0000 7fcf65688640 1 -- 172.21.15.50:0/1827620731 wait complete. 2024-04-02T16:24:34.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.908+0000 7fcf65688640 1 Processor -- start 2024-04-02T16:24:34.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.908+0000 7fcf65688640 1 -- start start 2024-04-02T16:24:34.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0039b0 0x7fcf60150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf60150960 0x7fcf600c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60150de0 0x7fcf600c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf600ccf70 con 0x7fcf60150de0 2024-04-02T16:24:34.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf600ccc90 con 0x7fcf60150960 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6006db50 con 0x7fcf4c0039b0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60150de0 0x7fcf600c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60150de0 0x7fcf600c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45060/0 (socket says 172.21.15.50:45060) 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 -- 172.21.15.50:0/1640559864 learned_addr learned my addr 172.21.15.50:0/1640559864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5e7fc640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf60150960 0x7fcf600c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0039b0 0x7fcf60150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 -- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0039b0 msgr2=0x7fcf60150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0039b0 0x7fcf60150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 -- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf60150960 msgr2=0x7fcf600c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf60150960 0x7fcf600c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 -- 172.21.15.50:0/1640559864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf600b4bf0 con 0x7fcf60150de0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0039b0 0x7fcf60150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf5f7fe640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60150de0 0x7fcf600c7fd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcf60072a10 tx=0x7fcf50007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf577fe640 1 -- 172.21.15.50:0/1640559864 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000ad00 con 0x7fcf60150de0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf577fe640 1 -- 172.21.15.50:0/1640559864 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50019070 con 0x7fcf60150de0 2024-04-02T16:24:34.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf577fe640 1 -- 172.21.15.50:0/1640559864 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50013630 con 0x7fcf60150de0 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf600b4e90 con 0x7fcf60150de0 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.909+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf600b7cb0 con 0x7fcf60150de0 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.911+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf600c8510 con 0x7fcf60150de0 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.911+0000 7fcf577fe640 1 -- 172.21.15.50:0/1640559864 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf500137d0 con 0x7fcf60150de0 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.911+0000 7fcf577fe640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcf4006c8f0 0x7fcf4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.911+0000 7fcf577fe640 1 -- 172.21.15.50:0/1640559864 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fcf5008c340 con 0x7fcf60150de0 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.912+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcf4006c8f0 0x7fcf4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:24:34.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.912+0000 7fcf5effd640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcf4006c8f0 0x7fcf4006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcf58036000 tx=0x7fcf58030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:24:34.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:34.916+0000 7fcf577fe640 1 -- 172.21.15.50:0/1640559864 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcf5005a5c0 con 0x7fcf60150de0 2024-04-02T16:24:35.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.168+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) -- 0x7fcf600b2480 con 0x7fcf60150de0 2024-04-02T16:24:35.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.169+0000 7fcf577fe640 1 -- 172.21.15.50:0/1640559864 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v9) ==== 92+0+63 (secure 0 0 0) 0x7fcf50059ec0 con 0x7fcf60150de0 2024-04-02T16:24:35.169 INFO:teuthology.orchestra.run.smithi050.stdout:[client.admin] 2024-04-02T16:24:35.169 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcf4006c8f0 msgr2=0x7fcf4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcf4006c8f0 0x7fcf4006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcf58036000 tx=0x7fcf58030000 comp rx=0 tx=0).stop 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60150de0 msgr2=0x7fcf600c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60150de0 0x7fcf600c7fd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcf60072a10 tx=0x7fcf50007bd0 comp rx=0 tx=0).stop 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 shutdown_connections 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7fcf4006c8f0 0x7fcf4006edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf60150de0 0x7fcf600c7fd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf60150960 0x7fcf600c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 --2- 172.21.15.50:0/1640559864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0039b0 0x7fcf60150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.170+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 >> 172.21.15.50:0/1640559864 conn(0x7fcf600b3ad0 msgr2=0x7fcf600b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.171+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 shutdown_connections 2024-04-02T16:24:35.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T16:24:35.171+0000 7fcf65688640 1 -- 172.21.15.50:0/1640559864 wait complete. 2024-04-02T16:24:35.182 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:24:35.182 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/tmp/tmp.CCuraAyba9xfstests-dev/local.config 2024-04-02T16:24:35.209 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.50:6789:/test export TEST_DIR=/tmp/tmp.rLt4hGA3Uftest export SCRATCH_DEV=172.21.15.50:6789:/scratch export SCRATCH_MNT=/tmp/tmp.9wSzMl1nX5scratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg==" 2024-04-02T16:24:35.209 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T16:24:35.209 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/tmp/tmp.CCuraAyba9xfstests-dev/ceph.exclude 2024-04-02T16:24:35.276 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo make) 2024-04-02T16:24:35.347 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize -cfi 2024-04-02T16:24:35.706 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-02T16:24:35.706 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file './config.guess' 2024-04-02T16:24:35.709 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file './config.sub' 2024-04-02T16:24:35.712 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file './install-sh' 2024-04-02T16:24:35.716 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file './ltmain.sh' 2024-04-02T16:24:35.737 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-02T16:24:35.737 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-02T16:24:35.741 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-02T16:24:35.745 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-02T16:24:35.750 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-02T16:24:35.754 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-02T16:24:35.760 INFO:teuthology.orchestra.run.smithi050.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-02T16:24:35.761 INFO:teuthology.orchestra.run.smithi050.stdout:cp include/install-sh . 2024-04-02T16:24:35.762 INFO:teuthology.orchestra.run.smithi050.stdout:aclocal -I m4 2024-04-02T16:24:36.794 INFO:teuthology.orchestra.run.smithi050.stdout:autoheader 2024-04-02T16:24:37.100 INFO:teuthology.orchestra.run.smithi050.stdout:autoconf 2024-04-02T16:24:37.272 INFO:teuthology.orchestra.run.smithi050.stdout:./configure \ 2024-04-02T16:24:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: --libexecdir=/usr/lib \ 2024-04-02T16:24:37.272 INFO:teuthology.orchestra.run.smithi050.stdout: --exec_prefix=/var/lib 2024-04-02T16:24:37.483 INFO:teuthology.orchestra.run.smithi050.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-02T16:24:37.483 INFO:teuthology.orchestra.run.smithi050.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-02T16:24:37.484 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to print strings... printf 2024-04-02T16:24:37.484 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gcc... gcc 2024-04-02T16:24:37.538 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether the C compiler works... yes 2024-04-02T16:24:37.538 INFO:teuthology.orchestra.run.smithi050.stdout:checking for C compiler default output file name... a.out 2024-04-02T16:24:37.565 INFO:teuthology.orchestra.run.smithi050.stdout:checking for suffix of executables... 2024-04-02T16:24:37.607 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether we are cross compiling... no 2024-04-02T16:24:37.622 INFO:teuthology.orchestra.run.smithi050.stdout:checking for suffix of object files... o 2024-04-02T16:24:37.637 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether we are using the GNU C compiler... yes 2024-04-02T16:24:37.652 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether gcc accepts -g... yes 2024-04-02T16:24:37.678 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-02T16:24:37.680 INFO:teuthology.orchestra.run.smithi050.stdout:checking for a sed that does not truncate output... /bin/sed 2024-04-02T16:24:37.682 INFO:teuthology.orchestra.run.smithi050.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-04-02T16:24:37.683 INFO:teuthology.orchestra.run.smithi050.stdout:checking for egrep... /bin/grep -E 2024-04-02T16:24:37.684 INFO:teuthology.orchestra.run.smithi050.stdout:checking for fgrep... /bin/grep -F 2024-04-02T16:24:37.688 INFO:teuthology.orchestra.run.smithi050.stdout:checking for ld used by gcc... /bin/ld 2024-04-02T16:24:37.689 INFO:teuthology.orchestra.run.smithi050.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-04-02T16:24:37.693 INFO:teuthology.orchestra.run.smithi050.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-04-02T16:24:38.654 INFO:teuthology.orchestra.run.smithi050.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-04-02T16:24:38.654 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether ln -s works... yes 2024-04-02T16:24:38.665 INFO:teuthology.orchestra.run.smithi050.stdout:checking the maximum length of command line arguments... 1572864 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking for /bin/ld option to reload object files... -r 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking for objdump... objdump 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to recognize dependent libraries... pass_all 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dlltool... no 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-02T16:24:38.669 INFO:teuthology.orchestra.run.smithi050.stdout:checking for ar... ar 2024-04-02T16:24:38.714 INFO:teuthology.orchestra.run.smithi050.stdout:checking for archiver @FILE support... @ 2024-04-02T16:24:38.714 INFO:teuthology.orchestra.run.smithi050.stdout:checking for strip... strip 2024-04-02T16:24:38.715 INFO:teuthology.orchestra.run.smithi050.stdout:checking for ranlib... ranlib 2024-04-02T16:24:38.715 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gawk... gawk 2024-04-02T16:24:38.780 INFO:teuthology.orchestra.run.smithi050.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-04-02T16:24:38.781 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sysroot... no 2024-04-02T16:24:38.784 INFO:teuthology.orchestra.run.smithi050.stdout:checking for a working dd... /bin/dd 2024-04-02T16:24:38.786 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-04-02T16:24:38.799 INFO:teuthology.orchestra.run.smithi050.stdout:checking for mt... no 2024-04-02T16:24:38.801 INFO:teuthology.orchestra.run.smithi050.stdout:checking if : is a manifest tool... no 2024-04-02T16:24:38.842 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to run the C preprocessor... gcc -E 2024-04-02T16:24:39.014 INFO:teuthology.orchestra.run.smithi050.stdout:checking for ANSI C header files... yes 2024-04-02T16:24:39.033 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/types.h... yes 2024-04-02T16:24:39.073 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/stat.h... yes 2024-04-02T16:24:39.096 INFO:teuthology.orchestra.run.smithi050.stdout:checking for stdlib.h... yes 2024-04-02T16:24:39.120 INFO:teuthology.orchestra.run.smithi050.stdout:checking for string.h... yes 2024-04-02T16:24:39.150 INFO:teuthology.orchestra.run.smithi050.stdout:checking for memory.h... yes 2024-04-02T16:24:39.174 INFO:teuthology.orchestra.run.smithi050.stdout:checking for strings.h... yes 2024-04-02T16:24:39.208 INFO:teuthology.orchestra.run.smithi050.stdout:checking for inttypes.h... yes 2024-04-02T16:24:39.232 INFO:teuthology.orchestra.run.smithi050.stdout:checking for stdint.h... yes 2024-04-02T16:24:39.265 INFO:teuthology.orchestra.run.smithi050.stdout:checking for unistd.h... yes 2024-04-02T16:24:39.293 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dlfcn.h... yes 2024-04-02T16:24:39.296 INFO:teuthology.orchestra.run.smithi050.stdout:checking for objdir... .libs 2024-04-02T16:24:39.357 INFO:teuthology.orchestra.run.smithi050.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-02T16:24:39.357 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-02T16:24:39.371 INFO:teuthology.orchestra.run.smithi050.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-02T16:24:39.388 INFO:teuthology.orchestra.run.smithi050.stdout:checking if gcc static flag -static works... no 2024-04-02T16:24:39.408 INFO:teuthology.orchestra.run.smithi050.stdout:checking if gcc supports -c -o file.o... yes 2024-04-02T16:24:39.408 INFO:teuthology.orchestra.run.smithi050.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-02T16:24:39.414 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-02T16:24:39.439 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether -lc should be explicitly linked in... no 2024-04-02T16:24:39.496 INFO:teuthology.orchestra.run.smithi050.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-02T16:24:39.496 INFO:teuthology.orchestra.run.smithi050.stdout:checking how to hardcode library paths into programs... immediate 2024-04-02T16:24:39.497 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether stripping libraries is possible... yes 2024-04-02T16:24:39.498 INFO:teuthology.orchestra.run.smithi050.stdout:checking if libtool supports shared libraries... yes 2024-04-02T16:24:39.498 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether to build shared libraries... yes 2024-04-02T16:24:39.498 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether to build static libraries... yes 2024-04-02T16:24:39.503 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gcc... (cached) gcc 2024-04-02T16:24:39.518 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-02T16:24:39.519 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-02T16:24:39.519 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-02T16:24:39.519 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gmake... /usr/bin/gmake 2024-04-02T16:24:39.519 INFO:teuthology.orchestra.run.smithi050.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-02T16:24:39.520 INFO:teuthology.orchestra.run.smithi050.stdout:checking for tar... /bin/tar 2024-04-02T16:24:39.520 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gzip... /bin/gzip 2024-04-02T16:24:39.527 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether gcc -MM is supported... yes 2024-04-02T16:24:39.527 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sort... /bin/sort 2024-04-02T16:24:39.528 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether ln -s works... yes 2024-04-02T16:24:39.528 INFO:teuthology.orchestra.run.smithi050.stdout:checking for rpm... /bin/rpm 2024-04-02T16:24:39.531 INFO:teuthology.orchestra.run.smithi050.stdout:checking for rpmbuild... no 2024-04-02T16:24:39.554 INFO:teuthology.orchestra.run.smithi050.stdout:checking assert.h usability... yes 2024-04-02T16:24:39.562 INFO:teuthology.orchestra.run.smithi050.stdout:checking assert.h presence... yes 2024-04-02T16:24:39.563 INFO:teuthology.orchestra.run.smithi050.stdout:checking for assert.h... yes 2024-04-02T16:24:39.588 INFO:teuthology.orchestra.run.smithi050.stdout:checking bstring.h usability... no 2024-04-02T16:24:39.598 INFO:teuthology.orchestra.run.smithi050.stdout:checking bstring.h presence... no 2024-04-02T16:24:39.598 INFO:teuthology.orchestra.run.smithi050.stdout:checking for bstring.h... no 2024-04-02T16:24:39.622 INFO:teuthology.orchestra.run.smithi050.stdout:checking libgen.h usability... yes 2024-04-02T16:24:39.630 INFO:teuthology.orchestra.run.smithi050.stdout:checking libgen.h presence... yes 2024-04-02T16:24:39.630 INFO:teuthology.orchestra.run.smithi050.stdout:checking for libgen.h... yes 2024-04-02T16:24:39.666 INFO:teuthology.orchestra.run.smithi050.stdout:checking dirent.h usability... yes 2024-04-02T16:24:39.675 INFO:teuthology.orchestra.run.smithi050.stdout:checking dirent.h presence... yes 2024-04-02T16:24:39.675 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dirent.h... yes 2024-04-02T16:24:39.713 INFO:teuthology.orchestra.run.smithi050.stdout:checking errno.h usability... yes 2024-04-02T16:24:39.721 INFO:teuthology.orchestra.run.smithi050.stdout:checking errno.h presence... yes 2024-04-02T16:24:39.721 INFO:teuthology.orchestra.run.smithi050.stdout:checking for errno.h... yes 2024-04-02T16:24:39.751 INFO:teuthology.orchestra.run.smithi050.stdout:checking malloc.h usability... yes 2024-04-02T16:24:39.761 INFO:teuthology.orchestra.run.smithi050.stdout:checking malloc.h presence... yes 2024-04-02T16:24:39.761 INFO:teuthology.orchestra.run.smithi050.stdout:checking for malloc.h... yes 2024-04-02T16:24:39.787 INFO:teuthology.orchestra.run.smithi050.stdout:checking uuid.h usability... no 2024-04-02T16:24:39.797 INFO:teuthology.orchestra.run.smithi050.stdout:checking uuid.h presence... no 2024-04-02T16:24:39.797 INFO:teuthology.orchestra.run.smithi050.stdout:checking for uuid.h... no 2024-04-02T16:24:39.855 INFO:teuthology.orchestra.run.smithi050.stdout:checking uuid/uuid.h usability... yes 2024-04-02T16:24:39.865 INFO:teuthology.orchestra.run.smithi050.stdout:checking uuid/uuid.h presence... yes 2024-04-02T16:24:39.865 INFO:teuthology.orchestra.run.smithi050.stdout:checking for uuid/uuid.h... yes 2024-04-02T16:24:39.890 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/uuid.h usability... no 2024-04-02T16:24:39.901 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/uuid.h presence... no 2024-04-02T16:24:39.901 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/uuid.h... no 2024-04-02T16:24:39.926 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/file.h usability... yes 2024-04-02T16:24:39.935 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/file.h presence... yes 2024-04-02T16:24:39.935 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/file.h... yes 2024-04-02T16:24:39.966 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/fcntl.h usability... yes 2024-04-02T16:24:39.975 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/fcntl.h presence... yes 2024-04-02T16:24:39.975 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/fcntl.h... yes 2024-04-02T16:24:40.001 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/syssgi.h usability... no 2024-04-02T16:24:40.011 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/syssgi.h presence... no 2024-04-02T16:24:40.011 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/syssgi.h... no 2024-04-02T16:24:40.052 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/param.h usability... yes 2024-04-02T16:24:40.063 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/param.h presence... yes 2024-04-02T16:24:40.063 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/param.h... yes 2024-04-02T16:24:40.067 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/stat.h... (cached) yes 2024-04-02T16:24:40.094 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/statvfs.h usability... yes 2024-04-02T16:24:40.102 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/statvfs.h presence... yes 2024-04-02T16:24:40.102 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/statvfs.h... yes 2024-04-02T16:24:40.128 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/time.h usability... yes 2024-04-02T16:24:40.137 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/time.h presence... yes 2024-04-02T16:24:40.137 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/time.h... yes 2024-04-02T16:24:40.177 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/ioctl.h usability... yes 2024-04-02T16:24:40.185 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/ioctl.h presence... yes 2024-04-02T16:24:40.185 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/ioctl.h... yes 2024-04-02T16:24:40.213 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/wait.h usability... yes 2024-04-02T16:24:40.226 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/wait.h presence... yes 2024-04-02T16:24:40.226 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/wait.h... yes 2024-04-02T16:24:40.228 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/types.h... (cached) yes 2024-04-02T16:24:40.232 INFO:teuthology.orchestra.run.smithi050.stdout:checking for strings.h... (cached) yes 2024-04-02T16:24:40.258 INFO:teuthology.orchestra.run.smithi050.stdout:checking err.h usability... yes 2024-04-02T16:24:40.267 INFO:teuthology.orchestra.run.smithi050.stdout:checking err.h presence... yes 2024-04-02T16:24:40.267 INFO:teuthology.orchestra.run.smithi050.stdout:checking for err.h... yes 2024-04-02T16:24:40.293 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/falloc.h usability... yes 2024-04-02T16:24:40.301 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/falloc.h presence... yes 2024-04-02T16:24:40.301 INFO:teuthology.orchestra.run.smithi050.stdout:checking for linux/falloc.h... yes 2024-04-02T16:24:40.327 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-02T16:24:40.337 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-02T16:24:40.337 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-02T16:24:40.361 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-02T16:24:40.371 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-02T16:24:40.371 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-02T16:24:40.394 INFO:teuthology.orchestra.run.smithi050.stdout:checking xfs/platform_defs.h usability... no 2024-04-02T16:24:40.405 INFO:teuthology.orchestra.run.smithi050.stdout:checking xfs/platform_defs.h presence... no 2024-04-02T16:24:40.405 INFO:teuthology.orchestra.run.smithi050.stdout:checking for xfs/platform_defs.h... no 2024-04-02T16:24:40.428 INFO:teuthology.orchestra.run.smithi050.stdout:checking btrfs/ioctl.h usability... no 2024-04-02T16:24:40.438 INFO:teuthology.orchestra.run.smithi050.stdout:checking btrfs/ioctl.h presence... no 2024-04-02T16:24:40.438 INFO:teuthology.orchestra.run.smithi050.stdout:checking for btrfs/ioctl.h... no 2024-04-02T16:24:40.461 INFO:teuthology.orchestra.run.smithi050.stdout:checking cifs/ioctl.h usability... no 2024-04-02T16:24:40.471 INFO:teuthology.orchestra.run.smithi050.stdout:checking cifs/ioctl.h presence... no 2024-04-02T16:24:40.471 INFO:teuthology.orchestra.run.smithi050.stdout:checking for cifs/ioctl.h... no 2024-04-02T16:24:40.496 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/mman.h usability... yes 2024-04-02T16:24:40.505 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/mman.h presence... yes 2024-04-02T16:24:40.505 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/mman.h... yes 2024-04-02T16:24:40.531 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/ext4.h usability... no 2024-04-02T16:24:40.541 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/ext4.h presence... no 2024-04-02T16:24:40.541 INFO:teuthology.orchestra.run.smithi050.stdout:checking for linux/ext4.h... no 2024-04-02T16:24:40.552 INFO:teuthology.orchestra.run.smithi050.stdout:checking for xfs/xfs_log_format.h... no 2024-04-02T16:24:40.586 INFO:teuthology.orchestra.run.smithi050.stdout:checking for uuid_compare... no 2024-04-02T16:24:40.616 INFO:teuthology.orchestra.run.smithi050.stdout:checking for uuid_compare in -luuid... yes 2024-04-02T16:24:40.677 INFO:teuthology.orchestra.run.smithi050.stdout:checking for xfs/xfs.h... yes 2024-04-02T16:24:40.690 INFO:teuthology.orchestra.run.smithi050.stdout:checking for xfs/libxfs.h... no 2024-04-02T16:24:40.704 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-02T16:24:40.733 INFO:teuthology.orchestra.run.smithi050.stdout:checking for xfs/xqm.h... yes 2024-04-02T16:24:40.780 INFO:teuthology.orchestra.run.smithi050.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-02T16:24:40.803 INFO:teuthology.orchestra.run.smithi050.stdout:checking xfs/handle.h usability... yes 2024-04-02T16:24:40.810 INFO:teuthology.orchestra.run.smithi050.stdout:checking xfs/handle.h presence... yes 2024-04-02T16:24:40.811 INFO:teuthology.orchestra.run.smithi050.stdout:checking for xfs/handle.h... yes 2024-04-02T16:24:40.841 INFO:teuthology.orchestra.run.smithi050.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-02T16:24:40.864 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/acl.h usability... yes 2024-04-02T16:24:40.874 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/acl.h presence... yes 2024-04-02T16:24:40.874 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/acl.h... yes 2024-04-02T16:24:40.897 INFO:teuthology.orchestra.run.smithi050.stdout:checking acl/libacl.h usability... yes 2024-04-02T16:24:40.906 INFO:teuthology.orchestra.run.smithi050.stdout:checking acl/libacl.h presence... yes 2024-04-02T16:24:40.906 INFO:teuthology.orchestra.run.smithi050.stdout:checking for acl/libacl.h... yes 2024-04-02T16:24:40.937 INFO:teuthology.orchestra.run.smithi050.stdout:checking for acl_init in -lacl... yes 2024-04-02T16:24:40.959 INFO:teuthology.orchestra.run.smithi050.stdout:checking gdbm-ndbm.h usability... no 2024-04-02T16:24:40.969 INFO:teuthology.orchestra.run.smithi050.stdout:checking gdbm-ndbm.h presence... no 2024-04-02T16:24:40.970 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gdbm-ndbm.h... no 2024-04-02T16:24:40.992 INFO:teuthology.orchestra.run.smithi050.stdout:checking gdbm/ndbm.h usability... yes 2024-04-02T16:24:41.001 INFO:teuthology.orchestra.run.smithi050.stdout:checking gdbm/ndbm.h presence... yes 2024-04-02T16:24:41.001 INFO:teuthology.orchestra.run.smithi050.stdout:checking for gdbm/ndbm.h... yes 2024-04-02T16:24:41.035 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_open in -lgdbm... no 2024-04-02T16:24:41.068 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_fetch in -lgdbm... no 2024-04-02T16:24:41.101 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_store in -lgdbm... no 2024-04-02T16:24:41.135 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_close in -lgdbm... no 2024-04-02T16:24:41.172 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-04-02T16:24:41.203 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-04-02T16:24:41.235 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-04-02T16:24:41.266 INFO:teuthology.orchestra.run.smithi050.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-04-02T16:24:41.289 INFO:teuthology.orchestra.run.smithi050.stdout:checking libaio.h usability... yes 2024-04-02T16:24:41.300 INFO:teuthology.orchestra.run.smithi050.stdout:checking libaio.h presence... yes 2024-04-02T16:24:41.300 INFO:teuthology.orchestra.run.smithi050.stdout:checking for libaio.h... yes 2024-04-02T16:24:41.322 INFO:teuthology.orchestra.run.smithi050.stdout:checking liburing.h usability... no 2024-04-02T16:24:41.332 INFO:teuthology.orchestra.run.smithi050.stdout:checking liburing.h presence... no 2024-04-02T16:24:41.332 INFO:teuthology.orchestra.run.smithi050.stdout:checking for liburing.h... no 2024-04-02T16:24:41.360 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/capability.h usability... yes 2024-04-02T16:24:41.370 INFO:teuthology.orchestra.run.smithi050.stdout:checking sys/capability.h presence... yes 2024-04-02T16:24:41.370 INFO:teuthology.orchestra.run.smithi050.stdout:checking for sys/capability.h... yes 2024-04-02T16:24:41.393 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/fiemap.h usability... yes 2024-04-02T16:24:41.401 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/fiemap.h presence... yes 2024-04-02T16:24:41.401 INFO:teuthology.orchestra.run.smithi050.stdout:checking for linux/fiemap.h... yes 2024-04-02T16:24:41.433 INFO:teuthology.orchestra.run.smithi050.stdout:checking for fallocate... yes 2024-04-02T16:24:41.466 INFO:teuthology.orchestra.run.smithi050.stdout:checking for open_by_handle_at... yes 2024-04-02T16:24:41.490 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/fs.h usability... yes 2024-04-02T16:24:41.499 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/fs.h presence... yes 2024-04-02T16:24:41.499 INFO:teuthology.orchestra.run.smithi050.stdout:checking for linux/fs.h... yes 2024-04-02T16:24:41.521 INFO:teuthology.orchestra.run.smithi050.stdout:checking btrfsutil.h usability... no 2024-04-02T16:24:41.531 INFO:teuthology.orchestra.run.smithi050.stdout:checking btrfsutil.h presence... no 2024-04-02T16:24:41.531 INFO:teuthology.orchestra.run.smithi050.stdout:checking for btrfsutil.h... no 2024-04-02T16:24:41.572 INFO:teuthology.orchestra.run.smithi050.stdout:checking for copy_file_range... yes 2024-04-02T16:24:41.606 INFO:teuthology.orchestra.run.smithi050.stdout:checking for SEEK_DATA... yes 2024-04-02T16:24:41.654 INFO:teuthology.orchestra.run.smithi050.stdout:checking for BMV_OF_SHARED... yes 2024-04-02T16:24:41.690 INFO:teuthology.orchestra.run.smithi050.stdout:checking for nftw... yes 2024-04-02T16:24:41.724 INFO:teuthology.orchestra.run.smithi050.stdout:checking for RLIMIT_NOFILE... yes 2024-04-02T16:24:41.755 INFO:teuthology.orchestra.run.smithi050.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-02T16:24:41.788 INFO:teuthology.orchestra.run.smithi050.stdout:checking for FICLONE... yes 2024-04-02T16:24:41.820 INFO:teuthology.orchestra.run.smithi050.stdout:checking for renameat2... yes 2024-04-02T16:24:41.853 INFO:teuthology.orchestra.run.smithi050.stdout:checking for reallocarray... yes 2024-04-02T16:24:41.882 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct mount_attr... yes 2024-04-02T16:24:41.914 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-02T16:24:41.945 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-02T16:24:41.976 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-02T16:24:42.008 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-02T16:24:42.039 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-02T16:24:42.070 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-02T16:24:42.102 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-02T16:24:42.128 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/btrfs.h usability... yes 2024-04-02T16:24:42.136 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/btrfs.h presence... yes 2024-04-02T16:24:42.136 INFO:teuthology.orchestra.run.smithi050.stdout:checking for linux/btrfs.h... yes 2024-04-02T16:24:42.165 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-02T16:24:42.174 INFO:teuthology.orchestra.run.smithi050.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-02T16:24:42.174 INFO:teuthology.orchestra.run.smithi050.stdout:checking for linux/btrfs_tree.h... yes 2024-04-02T16:24:42.194 INFO:teuthology.orchestra.run.smithi050.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-02T16:24:42.215 INFO:teuthology.orchestra.run.smithi050.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-02T16:24:42.231 INFO:teuthology.orchestra.run.smithi050.stdout:configure: creating ./config.status 2024-04-02T16:24:42.516 INFO:teuthology.orchestra.run.smithi050.stdout:config.status: creating include/builddefs 2024-04-02T16:24:42.526 INFO:teuthology.orchestra.run.smithi050.stdout:config.status: creating include/config.h 2024-04-02T16:24:42.534 INFO:teuthology.orchestra.run.smithi050.stdout:config.status: executing libtool commands 2024-04-02T16:24:42.570 INFO:teuthology.orchestra.run.smithi050.stdout:Building include 2024-04-02T16:24:42.573 INFO:teuthology.orchestra.run.smithi050.stdout:Building lib 2024-04-02T16:24:42.576 INFO:teuthology.orchestra.run.smithi050.stdout: [LTDEP] 2024-04-02T16:24:42.708 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dataascii.lo 2024-04-02T16:24:42.812 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] databin.lo 2024-04-02T16:24:42.949 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] datapid.lo 2024-04-02T16:24:43.042 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] file_lock.lo 2024-04-02T16:24:43.177 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] forker.lo 2024-04-02T16:24:43.296 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] pattern.lo 2024-04-02T16:24:43.405 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] open_flags.lo 2024-04-02T16:24:43.531 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] random_range.lo 2024-04-02T16:24:43.741 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] string_to_tokens.lo 2024-04-02T16:24:43.833 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] str_to_bytes.lo 2024-04-02T16:24:43.952 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] tlibio.lo 2024-04-02T16:24:44.170 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] write_log.lo 2024-04-02T16:24:44.388 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] random.lo 2024-04-02T16:24:44.486 INFO:teuthology.orchestra.run.smithi050.stdout: [LD] 2024-04-02T16:24:44.562 INFO:teuthology.orchestra.run.smithi050.stdout:Building ltp 2024-04-02T16:24:44.564 INFO:teuthology.orchestra.run.smithi050.stdout: [DEP] 2024-04-02T16:24:44.704 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] doio 2024-04-02T16:24:45.327 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fsstress 2024-04-02T16:24:46.695 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fsx 2024-04-02T16:24:47.387 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] growfiles 2024-04-02T16:24:47.847 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] iogen 2024-04-02T16:24:48.195 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-stress 2024-04-02T16:24:48.587 INFO:teuthology.orchestra.run.smithi050.stdout:Building src 2024-04-02T16:24:48.592 INFO:teuthology.orchestra.run.smithi050.stdout: [DEP] 2024-04-02T16:24:50.289 INFO:teuthology.orchestra.run.smithi050.stdout: [DEP] 2024-04-02T16:24:50.318 INFO:teuthology.orchestra.run.smithi050.stdout: [DEP] 2024-04-02T16:24:50.325 INFO:teuthology.orchestra.run.smithi050.stdout: [DEP] 2024-04-02T16:24:50.531 INFO:teuthology.orchestra.run.smithi050.stdout: [DEP] 2024-04-02T16:24:50.552 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dirstress 2024-04-02T16:24:50.761 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fill 2024-04-02T16:24:50.891 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fill2 2024-04-02T16:24:51.091 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] getpagesize 2024-04-02T16:24:51.203 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] holes 2024-04-02T16:24:51.370 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] lstat64 2024-04-02T16:24:51.526 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] nametest 2024-04-02T16:24:51.739 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] permname 2024-04-02T16:24:51.911 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] randholes 2024-04-02T16:24:52.128 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] runas 2024-04-02T16:24:52.276 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] truncfile 2024-04-02T16:24:52.430 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] usemem 2024-04-02T16:24:52.556 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] mmapcat 2024-04-02T16:24:52.681 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] append_reader 2024-04-02T16:24:52.807 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] append_writer 2024-04-02T16:24:52.933 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dirperf 2024-04-02T16:24:53.128 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] metaperf 2024-04-02T16:24:53.371 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] devzero 2024-04-02T16:24:53.521 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] feature 2024-04-02T16:24:53.710 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] alloc 2024-04-02T16:24:53.891 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fault 2024-04-02T16:24:54.026 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fstest 2024-04-02T16:24:54.232 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_access_root 2024-04-02T16:24:54.361 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] godown 2024-04-02T16:24:54.504 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] resvtest 2024-04-02T16:24:54.662 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] writemod 2024-04-02T16:24:54.784 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] writev_on_pagefault 2024-04-02T16:24:54.919 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] makeextents 2024-04-02T16:24:55.070 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] itrash 2024-04-02T16:24:55.199 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] rename 2024-04-02T16:24:55.315 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] multi_open_unlink 2024-04-02T16:24:55.454 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] unwritten_sync 2024-04-02T16:24:55.608 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] genhashnames 2024-04-02T16:24:55.772 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_holes 2024-04-02T16:24:55.896 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_writev 2024-04-02T16:24:56.020 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_truncate_cmtime 2024-04-02T16:24:56.152 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dirhash_collide 2024-04-02T16:24:56.310 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_rename_overwrite 2024-04-02T16:24:56.428 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] holetest 2024-04-02T16:24:56.628 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_truncate_self 2024-04-02T16:24:56.745 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] af_unix 2024-04-02T16:24:56.872 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_stale_pmd 2024-04-02T16:24:57.000 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_cow_race 2024-04-02T16:24:57.140 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_fallocate 2024-04-02T16:24:57.266 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fsync-err 2024-04-02T16:24:57.425 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_write_ro 2024-04-02T16:24:57.552 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-02T16:24:57.685 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-02T16:24:57.814 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_ofd_locks 2024-04-02T16:24:57.983 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_collision 2024-04-02T16:24:58.156 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] mmap-write-concurrent 2024-04-02T16:24:58.300 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_get_file_time 2024-04-02T16:24:58.423 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_create_short_dirs 2024-04-02T16:24:58.566 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_create_long_dirs 2024-04-02T16:24:58.708 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_enospc 2024-04-02T16:24:58.867 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_writev_overlap 2024-04-02T16:24:59.013 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] checkpoint_journal 2024-04-02T16:24:59.143 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] mmap-rw-fault 2024-04-02T16:24:59.300 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] allocstale 2024-04-02T16:24:59.443 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_cow_memory_failure 2024-04-02T16:24:59.588 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fake-dump-rootino 2024-04-02T16:24:59.714 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dio-buf-fault 2024-04-02T16:24:59.844 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] rewinddir-test 2024-04-02T16:24:59.988 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] readdir-while-renames 2024-04-02T16:25:00.128 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] xfsctl 2024-04-02T16:25:00.289 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] bstat 2024-04-02T16:25:00.468 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mtab 2024-04-02T16:25:00.629 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] getdevicesize 2024-04-02T16:25:00.748 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] preallo_rw_pattern_reader 2024-04-02T16:25:00.881 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] preallo_rw_pattern_writer 2024-04-02T16:25:01.013 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] ftrunc 2024-04-02T16:25:01.137 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] trunc 2024-04-02T16:25:01.274 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fs_perms 2024-04-02T16:25:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] testx 2024-04-02T16:25:01.523 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] looptest 2024-04-02T16:25:01.689 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] locktest 2024-04-02T16:25:01.965 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] unwritten_mmap 2024-04-02T16:25:02.103 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] bulkstat_unlink_test 2024-04-02T16:25:02.274 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] deduperace 2024-04-02T16:25:02.442 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] bulkstat_unlink_test_modified 2024-04-02T16:25:02.612 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_dir_offset 2024-04-02T16:25:02.737 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_futimens 2024-04-02T16:25:02.856 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_immutable 2024-04-02T16:25:02.900 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:25:02.902 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:25:03.436 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] stale_handle 2024-04-02T16:25:03.582 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] pwrite_mmap_blocked 2024-04-02T16:25:03.710 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_dir_offset2 2024-04-02T16:25:03.876 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] seek_sanity_test 2024-04-02T16:25:04.283 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] seek_copy_test 2024-04-02T16:25:04.434 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_readdir_1 2024-04-02T16:25:04.562 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_readdir_2 2024-04-02T16:25:04.686 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fsync-tester 2024-04-02T16:25:04.908 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] nsexec 2024-04-02T16:25:05.059 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] cloner 2024-04-02T16:25:05.211 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] renameat2 2024-04-02T16:25:05.351 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_getcwd 2024-04-02T16:25:05.485 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] e4compact 2024-04-02T16:25:05.662 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] test-nextquota 2024-04-02T16:25:05.808 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] punch-alternating 2024-04-02T16:25:05.953 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-02T16:25:06.094 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] listxattr 2024-04-02T16:25:06.219 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dio-interleaved 2024-04-02T16:25:06.355 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_dir_type 2024-04-02T16:25:06.489 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dio-invalidate-cache 2024-04-02T16:25:06.665 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] stat_test 2024-04-02T16:25:06.928 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_encrypted_d_revalidate 2024-04-02T16:25:07.069 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] attr_replace_test 2024-04-02T16:25:07.205 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] swapon 2024-04-02T16:25:07.345 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] mkswap 2024-04-02T16:25:07.474 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_attr_corruption 2024-04-02T16:25:07.601 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_open_tmpfiles 2024-04-02T16:25:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fscrypt-crypt-util 2024-04-02T16:25:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] bulkstat_null_ocount 2024-04-02T16:25:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] splice-test 2024-04-02T16:25:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] chprojid_fail 2024-04-02T16:25:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] detached_mounts_propagation 2024-04-02T16:25:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] ext4_resize 2024-04-02T16:25:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_readdir_3 2024-04-02T16:25:09.256 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] splice2pipe 2024-04-02T16:25:09.394 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] uuid_ioctl 2024-04-02T16:25:09.526 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-02T16:25:09.665 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fiemap-tester 2024-04-02T16:25:09.900 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_stripealign 2024-04-02T16:25:10.029 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] open_by_handle 2024-04-02T16:25:10.252 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] dbtest 2024-04-02T16:25:10.471 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_mmap_dio 2024-04-02T16:25:10.606 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] xfsfind 2024-04-02T16:25:10.775 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] t_reflink_read_race 2024-04-02T16:25:10.942 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] fssum 2024-04-02T16:25:11.346 INFO:teuthology.orchestra.run.smithi050.stdout:Building log-writes 2024-04-02T16:25:11.350 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] replay-log 2024-04-02T16:25:11.634 INFO:teuthology.orchestra.run.smithi050.stdout:Building perf 2024-04-02T16:25:11.637 INFO:teuthology.orchestra.run.smithi050.stdout:Building aio-dio-regress 2024-04-02T16:25:11.643 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aiocp 2024-04-02T16:25:11.848 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-02T16:25:12.047 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-append-write-read-race 2024-04-02T16:25:12.197 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-cow-race 2024-04-02T16:25:12.325 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-cycle-write 2024-04-02T16:25:12.484 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-eof-race 2024-04-02T16:25:12.641 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-extend-stat 2024-04-02T16:25:12.780 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-fcntl-race 2024-04-02T16:25:12.920 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-hole-filling-race 2024-04-02T16:25:13.045 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-invalidate-failure 2024-04-02T16:25:13.181 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-invalidate-readahead 2024-04-02T16:25:13.323 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aiodio_sparse2 2024-04-02T16:25:13.513 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-subblock-eof-read 2024-04-02T16:25:13.632 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-dio-write-verify 2024-04-02T16:25:13.801 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-02T16:25:13.914 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-02T16:25:14.025 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] aio-last-ref-held-by-io 2024-04-02T16:25:14.175 INFO:teuthology.orchestra.run.smithi050.stdout:Building vfs 2024-04-02T16:25:14.180 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] vfstest 2024-04-02T16:25:17.617 INFO:teuthology.orchestra.run.smithi050.stdout: [CC] mount-idmapped 2024-04-02T16:25:17.963 INFO:teuthology.orchestra.run.smithi050.stdout:Building m4 2024-04-02T16:25:17.966 INFO:teuthology.orchestra.run.smithi050.stdout:Building common 2024-04-02T16:25:17.969 INFO:teuthology.orchestra.run.smithi050.stdout:Building tests 2024-04-02T16:25:17.975 INFO:teuthology.orchestra.run.smithi050.stdout:Building btrfs 2024-04-02T16:25:17.980 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/btrfs/group.list 2024-04-02T16:25:18.038 INFO:teuthology.orchestra.run.smithi050.stdout:Building ceph 2024-04-02T16:25:18.043 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ceph/group.list 2024-04-02T16:25:18.057 INFO:teuthology.orchestra.run.smithi050.stdout:Building cifs 2024-04-02T16:25:18.062 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/cifs/group.list 2024-04-02T16:25:18.073 INFO:teuthology.orchestra.run.smithi050.stdout:Building ext4 2024-04-02T16:25:18.078 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ext4/group.list 2024-04-02T16:25:18.114 INFO:teuthology.orchestra.run.smithi050.stdout:Building f2fs 2024-04-02T16:25:18.119 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/f2fs/group.list 2024-04-02T16:25:18.135 INFO:teuthology.orchestra.run.smithi050.stdout:Building generic 2024-04-02T16:25:18.142 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/generic/group.list 2024-04-02T16:25:18.232 INFO:teuthology.orchestra.run.smithi050.stdout:Building nfs 2024-04-02T16:25:18.237 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/nfs/group.list 2024-04-02T16:25:18.251 INFO:teuthology.orchestra.run.smithi050.stdout:Building ocfs2 2024-04-02T16:25:18.255 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ocfs2/group.list 2024-04-02T16:25:18.268 INFO:teuthology.orchestra.run.smithi050.stdout:Building overlay 2024-04-02T16:25:18.273 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/overlay/group.list 2024-04-02T16:25:18.311 INFO:teuthology.orchestra.run.smithi050.stdout:Building perf 2024-04-02T16:25:18.316 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/perf/group.list 2024-04-02T16:25:18.327 INFO:teuthology.orchestra.run.smithi050.stdout:Building selftest 2024-04-02T16:25:18.332 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/selftest/group.list 2024-04-02T16:25:18.344 INFO:teuthology.orchestra.run.smithi050.stdout:Building shared 2024-04-02T16:25:18.348 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/shared/group.list 2024-04-02T16:25:18.365 INFO:teuthology.orchestra.run.smithi050.stdout:Building tmpfs 2024-04-02T16:25:18.369 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/tmpfs/group.list 2024-04-02T16:25:18.382 INFO:teuthology.orchestra.run.smithi050.stdout:Building udf 2024-04-02T16:25:18.387 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/udf/group.list 2024-04-02T16:25:18.397 INFO:teuthology.orchestra.run.smithi050.stdout:Building xfs 2024-04-02T16:25:18.403 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/xfs/group.list 2024-04-02T16:25:18.497 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo make install) 2024-04-02T16:25:18.534 INFO:teuthology.orchestra.run.smithi050.stdout:Building include 2024-04-02T16:25:18.537 INFO:teuthology.orchestra.run.smithi050.stdout:Building lib 2024-04-02T16:25:18.541 INFO:teuthology.orchestra.run.smithi050.stdout:Building ltp 2024-04-02T16:25:18.545 INFO:teuthology.orchestra.run.smithi050.stdout:Building src 2024-04-02T16:25:18.578 INFO:teuthology.orchestra.run.smithi050.stdout:Building log-writes 2024-04-02T16:25:18.581 INFO:teuthology.orchestra.run.smithi050.stdout:Building perf 2024-04-02T16:25:18.584 INFO:teuthology.orchestra.run.smithi050.stdout:Building aio-dio-regress 2024-04-02T16:25:18.588 INFO:teuthology.orchestra.run.smithi050.stdout:Building vfs 2024-04-02T16:25:18.592 INFO:teuthology.orchestra.run.smithi050.stdout:Building m4 2024-04-02T16:25:18.595 INFO:teuthology.orchestra.run.smithi050.stdout:Building common 2024-04-02T16:25:18.598 INFO:teuthology.orchestra.run.smithi050.stdout:Building tests 2024-04-02T16:25:18.603 INFO:teuthology.orchestra.run.smithi050.stdout:Building btrfs 2024-04-02T16:25:18.609 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/btrfs/group.list 2024-04-02T16:25:18.666 INFO:teuthology.orchestra.run.smithi050.stdout:Building ceph 2024-04-02T16:25:18.671 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ceph/group.list 2024-04-02T16:25:18.685 INFO:teuthology.orchestra.run.smithi050.stdout:Building cifs 2024-04-02T16:25:18.689 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/cifs/group.list 2024-04-02T16:25:18.701 INFO:teuthology.orchestra.run.smithi050.stdout:Building ext4 2024-04-02T16:25:18.706 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ext4/group.list 2024-04-02T16:25:18.742 INFO:teuthology.orchestra.run.smithi050.stdout:Building f2fs 2024-04-02T16:25:18.747 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/f2fs/group.list 2024-04-02T16:25:18.763 INFO:teuthology.orchestra.run.smithi050.stdout:Building generic 2024-04-02T16:25:18.769 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/generic/group.list 2024-04-02T16:25:18.859 INFO:teuthology.orchestra.run.smithi050.stdout:Building nfs 2024-04-02T16:25:18.864 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/nfs/group.list 2024-04-02T16:25:18.878 INFO:teuthology.orchestra.run.smithi050.stdout:Building ocfs2 2024-04-02T16:25:18.882 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ocfs2/group.list 2024-04-02T16:25:18.895 INFO:teuthology.orchestra.run.smithi050.stdout:Building overlay 2024-04-02T16:25:18.901 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/overlay/group.list 2024-04-02T16:25:18.938 INFO:teuthology.orchestra.run.smithi050.stdout:Building perf 2024-04-02T16:25:18.943 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/perf/group.list 2024-04-02T16:25:18.954 INFO:teuthology.orchestra.run.smithi050.stdout:Building selftest 2024-04-02T16:25:18.959 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/selftest/group.list 2024-04-02T16:25:18.970 INFO:teuthology.orchestra.run.smithi050.stdout:Building shared 2024-04-02T16:25:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/shared/group.list 2024-04-02T16:25:18.991 INFO:teuthology.orchestra.run.smithi050.stdout:Building tmpfs 2024-04-02T16:25:18.996 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/tmpfs/group.list 2024-04-02T16:25:19.009 INFO:teuthology.orchestra.run.smithi050.stdout:Building udf 2024-04-02T16:25:19.014 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/udf/group.list 2024-04-02T16:25:19.024 INFO:teuthology.orchestra.run.smithi050.stdout:Building xfs 2024-04-02T16:25:19.031 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/xfs/group.list 2024-04-02T16:25:19.120 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-02T16:25:19.122 INFO:teuthology.orchestra.run.smithi050.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-02T16:25:19.122 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-02T16:25:19.124 INFO:teuthology.orchestra.run.smithi050.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-02T16:25:19.124 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-02T16:25:19.126 INFO:teuthology.orchestra.run.smithi050.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-02T16:25:19.134 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-02T16:25:19.200 INFO:teuthology.orchestra.run.smithi050.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-02T16:25:19.208 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-02T16:25:19.224 INFO:teuthology.orchestra.run.smithi050.stdout:Building log-writes 2024-04-02T16:25:19.227 INFO:teuthology.orchestra.run.smithi050.stdout:Building perf 2024-04-02T16:25:19.230 INFO:teuthology.orchestra.run.smithi050.stdout:Building aio-dio-regress 2024-04-02T16:25:19.234 INFO:teuthology.orchestra.run.smithi050.stdout:Building vfs 2024-04-02T16:25:19.237 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake -C log-writes install 2024-04-02T16:25:19.239 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-02T16:25:19.247 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-02T16:25:19.255 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake -C perf install 2024-04-02T16:25:19.257 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-02T16:25:19.264 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-02T16:25:19.284 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-02T16:25:19.286 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-02T16:25:19.294 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-02T16:25:19.340 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake -C vfs install 2024-04-02T16:25:19.341 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-02T16:25:19.350 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-02T16:25:19.360 INFO:teuthology.orchestra.run.smithi050.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-02T16:25:19.368 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-02T16:25:20.462 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-02T16:25:20.552 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-02T16:25:20.575 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-02T16:25:20.577 INFO:teuthology.orchestra.run.smithi050.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-02T16:25:20.577 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-02T16:25:20.578 INFO:teuthology.orchestra.run.smithi050.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-02T16:25:20.586 INFO:teuthology.orchestra.run.smithi050.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-02T16:25:20.708 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-02T16:25:20.710 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/btrfs/ install 2024-04-02T16:25:20.714 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/btrfs/group.list 2024-04-02T16:25:20.771 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-02T16:25:20.779 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 /var/lib/xfstests/tests/btrfs 2024-04-02T16:25:21.532 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-02T16:25:21.540 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out /var/lib/xfstests/tests/btrfs 2024-04-02T16:25:22.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/ceph/ install 2024-04-02T16:25:22.296 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ceph/group.list 2024-04-02T16:25:22.309 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-02T16:25:22.317 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-02T16:25:22.334 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-02T16:25:22.342 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-02T16:25:22.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/cifs/ install 2024-04-02T16:25:22.362 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/cifs/group.list 2024-04-02T16:25:22.372 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-02T16:25:22.380 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-02T16:25:22.389 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-02T16:25:22.396 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-02T16:25:22.404 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/ext4/ install 2024-04-02T16:25:22.407 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ext4/group.list 2024-04-02T16:25:22.442 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-02T16:25:22.450 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-02T16:25:22.614 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-02T16:25:22.622 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-02T16:25:22.795 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/f2fs/ install 2024-04-02T16:25:22.797 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/f2fs/group.list 2024-04-02T16:25:22.812 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-02T16:25:22.820 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-02T16:25:22.831 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-02T16:25:22.839 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-02T16:25:22.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/generic/ install 2024-04-02T16:25:22.852 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/generic/group.list 2024-04-02T16:25:22.945 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-02T16:25:22.957 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 /var/lib/xfstests/tests/generic 2024-04-02T16:25:24.705 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-02T16:25:24.713 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out /var/lib/xfstests/tests/generic 2024-04-02T16:25:26.490 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/nfs/ install 2024-04-02T16:25:26.492 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/nfs/group.list 2024-04-02T16:25:26.505 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-02T16:25:26.513 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-02T16:25:26.521 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-02T16:25:26.529 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-02T16:25:26.538 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/ocfs2/ install 2024-04-02T16:25:26.540 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/ocfs2/group.list 2024-04-02T16:25:26.552 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-02T16:25:26.560 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-02T16:25:26.568 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-02T16:25:26.576 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-02T16:25:26.584 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/overlay/ install 2024-04-02T16:25:26.586 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/overlay/group.list 2024-04-02T16:25:26.624 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-02T16:25:26.632 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-02T16:25:26.883 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-02T16:25:26.891 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-02T16:25:27.140 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/perf/ install 2024-04-02T16:25:27.143 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/perf/group.list 2024-04-02T16:25:27.152 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-02T16:25:27.160 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-02T16:25:27.168 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-02T16:25:27.176 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-02T16:25:27.185 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/selftest/ install 2024-04-02T16:25:27.187 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/selftest/group.list 2024-04-02T16:25:27.198 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-02T16:25:27.206 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-02T16:25:27.226 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-02T16:25:27.233 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-02T16:25:27.253 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/shared/ install 2024-04-02T16:25:27.256 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/shared/group.list 2024-04-02T16:25:27.271 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-02T16:25:27.279 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-02T16:25:27.292 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-02T16:25:27.300 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-02T16:25:27.313 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/tmpfs/ install 2024-04-02T16:25:27.315 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/tmpfs/group.list 2024-04-02T16:25:27.327 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-02T16:25:27.335 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-02T16:25:27.343 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-02T16:25:27.351 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-02T16:25:27.359 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/udf/ install 2024-04-02T16:25:27.361 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/udf/group.list 2024-04-02T16:25:27.371 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-02T16:25:27.379 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-02T16:25:27.387 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-02T16:25:27.395 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-02T16:25:27.403 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CCuraAyba9xfstests-dev/tests/xfs/ install 2024-04-02T16:25:27.406 INFO:teuthology.orchestra.run.smithi050.stdout: [GROUP] /tmp/tmp.CCuraAyba9xfstests-dev/tests/xfs/group.list 2024-04-02T16:25:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-02T16:25:27.507 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-02T16:25:29.168 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-02T16:25:29.176 INFO:teuthology.orchestra.run.smithi050.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-02T16:25:30.861 INFO:teuthology.orchestra.run.smithi050.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-02T16:25:30.869 INFO:teuthology.orchestra.run.smithi050.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-02T16:25:30.877 INFO:teuthology.orchestra.run.smithi050.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-02T16:25:30.885 INFO:teuthology.orchestra.run.smithi050.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-02T16:25:30.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:25:30.895 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec ls tests/generic) 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:001 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:001.out 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:002 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:002.out 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:003 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:003.out 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:004 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:004.out 2024-04-02T16:25:30.952 INFO:teuthology.orchestra.run.smithi050.stdout:005 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:005.out 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:006 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:006.out 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:007 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:007.out 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:008 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:008.out 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:009 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:009.out 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:010 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:010.out 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:011 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:011.out 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:012 2024-04-02T16:25:30.953 INFO:teuthology.orchestra.run.smithi050.stdout:012.out 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:013 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:013.out 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:014 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:014.out 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:015 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:015.out 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:016 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:016.out 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:017 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:017.out 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:018 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:018.out 2024-04-02T16:25:30.954 INFO:teuthology.orchestra.run.smithi050.stdout:019 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:019.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:020 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:020.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:021 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:021.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:022 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:022.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:023 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:023.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:024 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:024.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:025 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:025.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:026 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:026.out 2024-04-02T16:25:30.955 INFO:teuthology.orchestra.run.smithi050.stdout:027 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:027.out 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:028 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:028.out 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:029 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:029.out 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:030 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:030.out 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:031 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:031.out 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:032 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:032.out 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:033 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:033.out 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:034 2024-04-02T16:25:30.956 INFO:teuthology.orchestra.run.smithi050.stdout:034.out 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:035 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:035.cfg 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:035.out.default 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:035.out.nfs 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:036 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:036.out 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:037 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:037.out 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:038 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:038.out 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:039 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:039.out 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:040 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:040.out 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:041 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:041.out 2024-04-02T16:25:30.957 INFO:teuthology.orchestra.run.smithi050.stdout:042 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:042.out 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:043 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:043.out 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:044 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:044.out 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:045 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:045.out 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:046 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:046.out 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:047 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:047.out 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:048 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:048.out 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:049 2024-04-02T16:25:30.958 INFO:teuthology.orchestra.run.smithi050.stdout:049.out 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:050 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:050.cfg 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:050.out.default 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:050.out.nojournal 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:050.out.xfsquota 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:051 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:051.out 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:052 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:052.out 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:053 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:053.out 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:054 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:054.out 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:055 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:055.out 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:056 2024-04-02T16:25:30.959 INFO:teuthology.orchestra.run.smithi050.stdout:056.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:057 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:057.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:058 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:058.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:059 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:059.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:060 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:060.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:061 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:061.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:062 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:062.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:063 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:063.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:064 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:064.out 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:065 2024-04-02T16:25:30.960 INFO:teuthology.orchestra.run.smithi050.stdout:065.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:066 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:066.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:067 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:067.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:068 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:068.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:069 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:069.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:070 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:070.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:071 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:071.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:072 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:072.out 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:073 2024-04-02T16:25:30.961 INFO:teuthology.orchestra.run.smithi050.stdout:073.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:074 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:074.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:075 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:075.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:076 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:076.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:077 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:077.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:078 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:078.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:079 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:079.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:080 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:080.out 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:081 2024-04-02T16:25:30.962 INFO:teuthology.orchestra.run.smithi050.stdout:081.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:082 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:082.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:083 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:083.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:084 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:084.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:085 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:085.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:086 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:086.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:087 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:087.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:088 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:088.out 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:089 2024-04-02T16:25:30.963 INFO:teuthology.orchestra.run.smithi050.stdout:089.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:090 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:090.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:091 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:091.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:092 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:092.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:093 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:093.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:094 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:094.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:095 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:095.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:096 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:096.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:097 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:097.out 2024-04-02T16:25:30.964 INFO:teuthology.orchestra.run.smithi050.stdout:098 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:098.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:099 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:099.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:100 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:100.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:101 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:101.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:102 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:102.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:103 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:103.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:104 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:104.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:105 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:105.out 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:106 2024-04-02T16:25:30.965 INFO:teuthology.orchestra.run.smithi050.stdout:106.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:107 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:107.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:108 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:108.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:109 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:109.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:110 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:110.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:111 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:111.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:112 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:112.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:113 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:113.out 2024-04-02T16:25:30.966 INFO:teuthology.orchestra.run.smithi050.stdout:114 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:114.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:115 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:115.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:116 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:116.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:117 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:117.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:118 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:118.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:119 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:119.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:120 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:120.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:121 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:121.out 2024-04-02T16:25:30.967 INFO:teuthology.orchestra.run.smithi050.stdout:122 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:122.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:123 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:123.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:124 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:124.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:125 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:125.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:126 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:126.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:127 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:127.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:128 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:128.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:129 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:129.out 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:130 2024-04-02T16:25:30.968 INFO:teuthology.orchestra.run.smithi050.stdout:130.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:131 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:131.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:132 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:132.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:133 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:133.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:134 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:134.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:135 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:135.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:136 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:136.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:137 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:137.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:138 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:138.out 2024-04-02T16:25:30.969 INFO:teuthology.orchestra.run.smithi050.stdout:139 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:139.out 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:140 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:140.out 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:141 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:141.out 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:142 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:142.out 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:143 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:143.out 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:144 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:144.out 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:145 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:145.out 2024-04-02T16:25:30.970 INFO:teuthology.orchestra.run.smithi050.stdout:146 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:146.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:147 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:147.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:148 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:148.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:149 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:149.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:150 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:150.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:151 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:151.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:152 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:152.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:153 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:153.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:154 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:154.out 2024-04-02T16:25:30.971 INFO:teuthology.orchestra.run.smithi050.stdout:155 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:155.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:156 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:156.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:157 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:157.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:158 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:158.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:159 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:159.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:160 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:160.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:161 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:161.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:162 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:162.out 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:163 2024-04-02T16:25:30.972 INFO:teuthology.orchestra.run.smithi050.stdout:163.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:164 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:164.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:165 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:165.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:166 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:166.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:167 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:167.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:168 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:168.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:169 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:169.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:170 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:170.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:171 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:171.out 2024-04-02T16:25:30.973 INFO:teuthology.orchestra.run.smithi050.stdout:172 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:172.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:173 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:173.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:174 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:174.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:175 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:175.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:176 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:176.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:177 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:177.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:178 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:178.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:179 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:179.out 2024-04-02T16:25:30.974 INFO:teuthology.orchestra.run.smithi050.stdout:180 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:180.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:181 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:181.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:182 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:182.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:183 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:183.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:184 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:184.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:185 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:185.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:186 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:186.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:187 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:187.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:188 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:188.out 2024-04-02T16:25:30.975 INFO:teuthology.orchestra.run.smithi050.stdout:189 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:189.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:190 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:190.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:191 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:191.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:192 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:192.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:193 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:193.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:194 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:194.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:195 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:195.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:196 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:196.out 2024-04-02T16:25:30.976 INFO:teuthology.orchestra.run.smithi050.stdout:197 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:197.out 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:198 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:198.out 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:199 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:199.out 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:200 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:200.out 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:201 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:201.out 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:202 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:202.out 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:203 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:203.out 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:204 2024-04-02T16:25:30.977 INFO:teuthology.orchestra.run.smithi050.stdout:204.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:205 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:205.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:206 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:206.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:207 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:207.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:208 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:208.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:209 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:209.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:210 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:210.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:211 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:211.out 2024-04-02T16:25:30.978 INFO:teuthology.orchestra.run.smithi050.stdout:212 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:212.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:213 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:213.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:214 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:214.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:215 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:215.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:216 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:216.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:217 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:217.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:218 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:218.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:219 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:219.out 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:220 2024-04-02T16:25:30.979 INFO:teuthology.orchestra.run.smithi050.stdout:220.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:221 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:221.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:222 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:222.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:223 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:223.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:224 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:224.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:225 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:225.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:226 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:226.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:227 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:227.out 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:228 2024-04-02T16:25:30.980 INFO:teuthology.orchestra.run.smithi050.stdout:228.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:229 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:229.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:230 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:230.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:231 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:231.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:232 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:232.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:233 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:233.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:234 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:234.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:235 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:235.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:236 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:236.out 2024-04-02T16:25:30.981 INFO:teuthology.orchestra.run.smithi050.stdout:237 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:237.out 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:238 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:238.out 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:239 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:239.out 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:240 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:240.out 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:241 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:241.out 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:242 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:242.out 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:243 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:243.out 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:244 2024-04-02T16:25:30.982 INFO:teuthology.orchestra.run.smithi050.stdout:244.out 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:245 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:245.out 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:246 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:246.out 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:247 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:247.out 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:248 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:248.out 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:249 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:249.out 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:250 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:250.out 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:251 2024-04-02T16:25:30.983 INFO:teuthology.orchestra.run.smithi050.stdout:251.out 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:252 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:252.out 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:253 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:253.out 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:254 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:254.out 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:255 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:255.out 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:256 2024-04-02T16:25:30.984 INFO:teuthology.orchestra.run.smithi050.stdout:256.out 2024-04-02T16:25:30.985 INFO:teuthology.orchestra.run.smithi050.stdout:257 2024-04-02T16:25:30.985 INFO:teuthology.orchestra.run.smithi050.stdout:257.out 2024-04-02T16:25:30.985 INFO:teuthology.orchestra.run.smithi050.stdout:258 2024-04-02T16:25:30.985 INFO:teuthology.orchestra.run.smithi050.stdout:258.out 2024-04-02T16:25:30.985 INFO:teuthology.orchestra.run.smithi050.stdout:259 2024-04-02T16:25:30.985 INFO:teuthology.orchestra.run.smithi050.stdout:259.out 2024-04-02T16:25:30.985 INFO:teuthology.orchestra.run.smithi050.stdout:260 2024-04-02T16:25:30.986 INFO:teuthology.orchestra.run.smithi050.stdout:260.out 2024-04-02T16:25:30.986 INFO:teuthology.orchestra.run.smithi050.stdout:261 2024-04-02T16:25:30.986 INFO:teuthology.orchestra.run.smithi050.stdout:261.out 2024-04-02T16:25:30.986 INFO:teuthology.orchestra.run.smithi050.stdout:262 2024-04-02T16:25:30.986 INFO:teuthology.orchestra.run.smithi050.stdout:262.out 2024-04-02T16:25:30.986 INFO:teuthology.orchestra.run.smithi050.stdout:263 2024-04-02T16:25:30.987 INFO:teuthology.orchestra.run.smithi050.stdout:263.out 2024-04-02T16:25:30.987 INFO:teuthology.orchestra.run.smithi050.stdout:264 2024-04-02T16:25:30.987 INFO:teuthology.orchestra.run.smithi050.stdout:264.out 2024-04-02T16:25:30.987 INFO:teuthology.orchestra.run.smithi050.stdout:265 2024-04-02T16:25:30.987 INFO:teuthology.orchestra.run.smithi050.stdout:265.out 2024-04-02T16:25:30.987 INFO:teuthology.orchestra.run.smithi050.stdout:266 2024-04-02T16:25:30.988 INFO:teuthology.orchestra.run.smithi050.stdout:266.out 2024-04-02T16:25:30.988 INFO:teuthology.orchestra.run.smithi050.stdout:267 2024-04-02T16:25:30.988 INFO:teuthology.orchestra.run.smithi050.stdout:267.out 2024-04-02T16:25:30.988 INFO:teuthology.orchestra.run.smithi050.stdout:268 2024-04-02T16:25:30.988 INFO:teuthology.orchestra.run.smithi050.stdout:268.out 2024-04-02T16:25:30.988 INFO:teuthology.orchestra.run.smithi050.stdout:269 2024-04-02T16:25:30.988 INFO:teuthology.orchestra.run.smithi050.stdout:269.out 2024-04-02T16:25:30.989 INFO:teuthology.orchestra.run.smithi050.stdout:270 2024-04-02T16:25:30.989 INFO:teuthology.orchestra.run.smithi050.stdout:270.out 2024-04-02T16:25:30.989 INFO:teuthology.orchestra.run.smithi050.stdout:271 2024-04-02T16:25:30.989 INFO:teuthology.orchestra.run.smithi050.stdout:271.out 2024-04-02T16:25:30.989 INFO:teuthology.orchestra.run.smithi050.stdout:272 2024-04-02T16:25:30.989 INFO:teuthology.orchestra.run.smithi050.stdout:272.out 2024-04-02T16:25:30.989 INFO:teuthology.orchestra.run.smithi050.stdout:273 2024-04-02T16:25:30.990 INFO:teuthology.orchestra.run.smithi050.stdout:273.out 2024-04-02T16:25:30.990 INFO:teuthology.orchestra.run.smithi050.stdout:274 2024-04-02T16:25:30.990 INFO:teuthology.orchestra.run.smithi050.stdout:274.out 2024-04-02T16:25:30.990 INFO:teuthology.orchestra.run.smithi050.stdout:275 2024-04-02T16:25:30.990 INFO:teuthology.orchestra.run.smithi050.stdout:275.out 2024-04-02T16:25:30.990 INFO:teuthology.orchestra.run.smithi050.stdout:276 2024-04-02T16:25:30.991 INFO:teuthology.orchestra.run.smithi050.stdout:276.out 2024-04-02T16:25:30.991 INFO:teuthology.orchestra.run.smithi050.stdout:277 2024-04-02T16:25:30.991 INFO:teuthology.orchestra.run.smithi050.stdout:277.out 2024-04-02T16:25:30.991 INFO:teuthology.orchestra.run.smithi050.stdout:278 2024-04-02T16:25:30.991 INFO:teuthology.orchestra.run.smithi050.stdout:278.out 2024-04-02T16:25:30.991 INFO:teuthology.orchestra.run.smithi050.stdout:279 2024-04-02T16:25:30.991 INFO:teuthology.orchestra.run.smithi050.stdout:279.out 2024-04-02T16:25:30.992 INFO:teuthology.orchestra.run.smithi050.stdout:280 2024-04-02T16:25:30.992 INFO:teuthology.orchestra.run.smithi050.stdout:280.out 2024-04-02T16:25:30.992 INFO:teuthology.orchestra.run.smithi050.stdout:281 2024-04-02T16:25:30.992 INFO:teuthology.orchestra.run.smithi050.stdout:281.out 2024-04-02T16:25:30.992 INFO:teuthology.orchestra.run.smithi050.stdout:282 2024-04-02T16:25:30.992 INFO:teuthology.orchestra.run.smithi050.stdout:282.out 2024-04-02T16:25:30.993 INFO:teuthology.orchestra.run.smithi050.stdout:283 2024-04-02T16:25:30.993 INFO:teuthology.orchestra.run.smithi050.stdout:283.out 2024-04-02T16:25:30.993 INFO:teuthology.orchestra.run.smithi050.stdout:284 2024-04-02T16:25:30.993 INFO:teuthology.orchestra.run.smithi050.stdout:284.out 2024-04-02T16:25:30.993 INFO:teuthology.orchestra.run.smithi050.stdout:285 2024-04-02T16:25:30.993 INFO:teuthology.orchestra.run.smithi050.stdout:285.out 2024-04-02T16:25:30.993 INFO:teuthology.orchestra.run.smithi050.stdout:286 2024-04-02T16:25:30.994 INFO:teuthology.orchestra.run.smithi050.stdout:286.out 2024-04-02T16:25:30.994 INFO:teuthology.orchestra.run.smithi050.stdout:287 2024-04-02T16:25:30.994 INFO:teuthology.orchestra.run.smithi050.stdout:287.out 2024-04-02T16:25:30.994 INFO:teuthology.orchestra.run.smithi050.stdout:288 2024-04-02T16:25:30.994 INFO:teuthology.orchestra.run.smithi050.stdout:288.out 2024-04-02T16:25:30.994 INFO:teuthology.orchestra.run.smithi050.stdout:289 2024-04-02T16:25:30.995 INFO:teuthology.orchestra.run.smithi050.stdout:289.out 2024-04-02T16:25:30.995 INFO:teuthology.orchestra.run.smithi050.stdout:290 2024-04-02T16:25:30.995 INFO:teuthology.orchestra.run.smithi050.stdout:290.out 2024-04-02T16:25:30.995 INFO:teuthology.orchestra.run.smithi050.stdout:291 2024-04-02T16:25:30.995 INFO:teuthology.orchestra.run.smithi050.stdout:291.out 2024-04-02T16:25:30.995 INFO:teuthology.orchestra.run.smithi050.stdout:292 2024-04-02T16:25:30.995 INFO:teuthology.orchestra.run.smithi050.stdout:292.out 2024-04-02T16:25:30.996 INFO:teuthology.orchestra.run.smithi050.stdout:293 2024-04-02T16:25:30.996 INFO:teuthology.orchestra.run.smithi050.stdout:293.out 2024-04-02T16:25:30.996 INFO:teuthology.orchestra.run.smithi050.stdout:294 2024-04-02T16:25:30.996 INFO:teuthology.orchestra.run.smithi050.stdout:294.out 2024-04-02T16:25:30.996 INFO:teuthology.orchestra.run.smithi050.stdout:295 2024-04-02T16:25:30.996 INFO:teuthology.orchestra.run.smithi050.stdout:295.out 2024-04-02T16:25:30.997 INFO:teuthology.orchestra.run.smithi050.stdout:296 2024-04-02T16:25:30.997 INFO:teuthology.orchestra.run.smithi050.stdout:296.out 2024-04-02T16:25:30.997 INFO:teuthology.orchestra.run.smithi050.stdout:297 2024-04-02T16:25:30.997 INFO:teuthology.orchestra.run.smithi050.stdout:297.out 2024-04-02T16:25:30.997 INFO:teuthology.orchestra.run.smithi050.stdout:298 2024-04-02T16:25:30.997 INFO:teuthology.orchestra.run.smithi050.stdout:298.out 2024-04-02T16:25:30.997 INFO:teuthology.orchestra.run.smithi050.stdout:299 2024-04-02T16:25:30.998 INFO:teuthology.orchestra.run.smithi050.stdout:299.out 2024-04-02T16:25:30.998 INFO:teuthology.orchestra.run.smithi050.stdout:300 2024-04-02T16:25:30.998 INFO:teuthology.orchestra.run.smithi050.stdout:300.out 2024-04-02T16:25:30.998 INFO:teuthology.orchestra.run.smithi050.stdout:301 2024-04-02T16:25:30.998 INFO:teuthology.orchestra.run.smithi050.stdout:301.out 2024-04-02T16:25:30.998 INFO:teuthology.orchestra.run.smithi050.stdout:302 2024-04-02T16:25:30.999 INFO:teuthology.orchestra.run.smithi050.stdout:302.out 2024-04-02T16:25:30.999 INFO:teuthology.orchestra.run.smithi050.stdout:303 2024-04-02T16:25:30.999 INFO:teuthology.orchestra.run.smithi050.stdout:303.out 2024-04-02T16:25:30.999 INFO:teuthology.orchestra.run.smithi050.stdout:304 2024-04-02T16:25:30.999 INFO:teuthology.orchestra.run.smithi050.stdout:304.out 2024-04-02T16:25:30.999 INFO:teuthology.orchestra.run.smithi050.stdout:305 2024-04-02T16:25:31.000 INFO:teuthology.orchestra.run.smithi050.stdout:305.out 2024-04-02T16:25:31.000 INFO:teuthology.orchestra.run.smithi050.stdout:306 2024-04-02T16:25:31.000 INFO:teuthology.orchestra.run.smithi050.stdout:306.out 2024-04-02T16:25:31.000 INFO:teuthology.orchestra.run.smithi050.stdout:307 2024-04-02T16:25:31.000 INFO:teuthology.orchestra.run.smithi050.stdout:307.out 2024-04-02T16:25:31.000 INFO:teuthology.orchestra.run.smithi050.stdout:308 2024-04-02T16:25:31.000 INFO:teuthology.orchestra.run.smithi050.stdout:308.out 2024-04-02T16:25:31.001 INFO:teuthology.orchestra.run.smithi050.stdout:309 2024-04-02T16:25:31.001 INFO:teuthology.orchestra.run.smithi050.stdout:309.out 2024-04-02T16:25:31.001 INFO:teuthology.orchestra.run.smithi050.stdout:310 2024-04-02T16:25:31.001 INFO:teuthology.orchestra.run.smithi050.stdout:310.out 2024-04-02T16:25:31.001 INFO:teuthology.orchestra.run.smithi050.stdout:311 2024-04-02T16:25:31.001 INFO:teuthology.orchestra.run.smithi050.stdout:311.out 2024-04-02T16:25:31.001 INFO:teuthology.orchestra.run.smithi050.stdout:312 2024-04-02T16:25:31.002 INFO:teuthology.orchestra.run.smithi050.stdout:312.out 2024-04-02T16:25:31.002 INFO:teuthology.orchestra.run.smithi050.stdout:313 2024-04-02T16:25:31.002 INFO:teuthology.orchestra.run.smithi050.stdout:313.out 2024-04-02T16:25:31.002 INFO:teuthology.orchestra.run.smithi050.stdout:314 2024-04-02T16:25:31.002 INFO:teuthology.orchestra.run.smithi050.stdout:314.out 2024-04-02T16:25:31.002 INFO:teuthology.orchestra.run.smithi050.stdout:315 2024-04-02T16:25:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:315.out 2024-04-02T16:25:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:316 2024-04-02T16:25:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:316.out 2024-04-02T16:25:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:317 2024-04-02T16:25:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:317.out 2024-04-02T16:25:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:318 2024-04-02T16:25:31.003 INFO:teuthology.orchestra.run.smithi050.stdout:318.out 2024-04-02T16:25:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:319 2024-04-02T16:25:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:319.out 2024-04-02T16:25:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:320 2024-04-02T16:25:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:320.out 2024-04-02T16:25:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:321 2024-04-02T16:25:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:321.out 2024-04-02T16:25:31.005 INFO:teuthology.orchestra.run.smithi050.stdout:322 2024-04-02T16:25:31.005 INFO:teuthology.orchestra.run.smithi050.stdout:322.out 2024-04-02T16:25:31.005 INFO:teuthology.orchestra.run.smithi050.stdout:323 2024-04-02T16:25:31.005 INFO:teuthology.orchestra.run.smithi050.stdout:323.out 2024-04-02T16:25:31.005 INFO:teuthology.orchestra.run.smithi050.stdout:324 2024-04-02T16:25:31.005 INFO:teuthology.orchestra.run.smithi050.stdout:324.out 2024-04-02T16:25:31.006 INFO:teuthology.orchestra.run.smithi050.stdout:325 2024-04-02T16:25:31.006 INFO:teuthology.orchestra.run.smithi050.stdout:325.out 2024-04-02T16:25:31.006 INFO:teuthology.orchestra.run.smithi050.stdout:326 2024-04-02T16:25:31.006 INFO:teuthology.orchestra.run.smithi050.stdout:326.out 2024-04-02T16:25:31.006 INFO:teuthology.orchestra.run.smithi050.stdout:327 2024-04-02T16:25:31.006 INFO:teuthology.orchestra.run.smithi050.stdout:327.out 2024-04-02T16:25:31.007 INFO:teuthology.orchestra.run.smithi050.stdout:328 2024-04-02T16:25:31.007 INFO:teuthology.orchestra.run.smithi050.stdout:328.out 2024-04-02T16:25:31.007 INFO:teuthology.orchestra.run.smithi050.stdout:329 2024-04-02T16:25:31.007 INFO:teuthology.orchestra.run.smithi050.stdout:329.out 2024-04-02T16:25:31.007 INFO:teuthology.orchestra.run.smithi050.stdout:330 2024-04-02T16:25:31.007 INFO:teuthology.orchestra.run.smithi050.stdout:330.out 2024-04-02T16:25:31.007 INFO:teuthology.orchestra.run.smithi050.stdout:331 2024-04-02T16:25:31.008 INFO:teuthology.orchestra.run.smithi050.stdout:331.out 2024-04-02T16:25:31.008 INFO:teuthology.orchestra.run.smithi050.stdout:332 2024-04-02T16:25:31.008 INFO:teuthology.orchestra.run.smithi050.stdout:332.out 2024-04-02T16:25:31.008 INFO:teuthology.orchestra.run.smithi050.stdout:333 2024-04-02T16:25:31.008 INFO:teuthology.orchestra.run.smithi050.stdout:333.out 2024-04-02T16:25:31.008 INFO:teuthology.orchestra.run.smithi050.stdout:334 2024-04-02T16:25:31.009 INFO:teuthology.orchestra.run.smithi050.stdout:334.out 2024-04-02T16:25:31.009 INFO:teuthology.orchestra.run.smithi050.stdout:335 2024-04-02T16:25:31.009 INFO:teuthology.orchestra.run.smithi050.stdout:335.out 2024-04-02T16:25:31.009 INFO:teuthology.orchestra.run.smithi050.stdout:336 2024-04-02T16:25:31.009 INFO:teuthology.orchestra.run.smithi050.stdout:336.out 2024-04-02T16:25:31.009 INFO:teuthology.orchestra.run.smithi050.stdout:337 2024-04-02T16:25:31.009 INFO:teuthology.orchestra.run.smithi050.stdout:337.out 2024-04-02T16:25:31.010 INFO:teuthology.orchestra.run.smithi050.stdout:338 2024-04-02T16:25:31.010 INFO:teuthology.orchestra.run.smithi050.stdout:338.out 2024-04-02T16:25:31.010 INFO:teuthology.orchestra.run.smithi050.stdout:339 2024-04-02T16:25:31.010 INFO:teuthology.orchestra.run.smithi050.stdout:339.out 2024-04-02T16:25:31.010 INFO:teuthology.orchestra.run.smithi050.stdout:340 2024-04-02T16:25:31.010 INFO:teuthology.orchestra.run.smithi050.stdout:340.out 2024-04-02T16:25:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:341 2024-04-02T16:25:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:341.out 2024-04-02T16:25:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:342 2024-04-02T16:25:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:342.out 2024-04-02T16:25:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:343 2024-04-02T16:25:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:343.out 2024-04-02T16:25:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:344 2024-04-02T16:25:31.012 INFO:teuthology.orchestra.run.smithi050.stdout:344.out 2024-04-02T16:25:31.012 INFO:teuthology.orchestra.run.smithi050.stdout:345 2024-04-02T16:25:31.012 INFO:teuthology.orchestra.run.smithi050.stdout:345.out 2024-04-02T16:25:31.012 INFO:teuthology.orchestra.run.smithi050.stdout:346 2024-04-02T16:25:31.012 INFO:teuthology.orchestra.run.smithi050.stdout:346.out 2024-04-02T16:25:31.012 INFO:teuthology.orchestra.run.smithi050.stdout:347 2024-04-02T16:25:31.013 INFO:teuthology.orchestra.run.smithi050.stdout:347.out 2024-04-02T16:25:31.013 INFO:teuthology.orchestra.run.smithi050.stdout:348 2024-04-02T16:25:31.013 INFO:teuthology.orchestra.run.smithi050.stdout:348.out 2024-04-02T16:25:31.013 INFO:teuthology.orchestra.run.smithi050.stdout:349 2024-04-02T16:25:31.013 INFO:teuthology.orchestra.run.smithi050.stdout:349.out 2024-04-02T16:25:31.013 INFO:teuthology.orchestra.run.smithi050.stdout:350 2024-04-02T16:25:31.014 INFO:teuthology.orchestra.run.smithi050.stdout:350.out 2024-04-02T16:25:31.014 INFO:teuthology.orchestra.run.smithi050.stdout:351 2024-04-02T16:25:31.014 INFO:teuthology.orchestra.run.smithi050.stdout:351.out 2024-04-02T16:25:31.014 INFO:teuthology.orchestra.run.smithi050.stdout:352 2024-04-02T16:25:31.014 INFO:teuthology.orchestra.run.smithi050.stdout:352.out 2024-04-02T16:25:31.014 INFO:teuthology.orchestra.run.smithi050.stdout:353 2024-04-02T16:25:31.015 INFO:teuthology.orchestra.run.smithi050.stdout:353.out 2024-04-02T16:25:31.015 INFO:teuthology.orchestra.run.smithi050.stdout:354 2024-04-02T16:25:31.015 INFO:teuthology.orchestra.run.smithi050.stdout:354.out 2024-04-02T16:25:31.015 INFO:teuthology.orchestra.run.smithi050.stdout:355 2024-04-02T16:25:31.015 INFO:teuthology.orchestra.run.smithi050.stdout:355.out 2024-04-02T16:25:31.015 INFO:teuthology.orchestra.run.smithi050.stdout:356 2024-04-02T16:25:31.015 INFO:teuthology.orchestra.run.smithi050.stdout:356.out 2024-04-02T16:25:31.016 INFO:teuthology.orchestra.run.smithi050.stdout:357 2024-04-02T16:25:31.016 INFO:teuthology.orchestra.run.smithi050.stdout:357.out 2024-04-02T16:25:31.016 INFO:teuthology.orchestra.run.smithi050.stdout:358 2024-04-02T16:25:31.016 INFO:teuthology.orchestra.run.smithi050.stdout:358.out 2024-04-02T16:25:31.016 INFO:teuthology.orchestra.run.smithi050.stdout:359 2024-04-02T16:25:31.016 INFO:teuthology.orchestra.run.smithi050.stdout:359.out 2024-04-02T16:25:31.017 INFO:teuthology.orchestra.run.smithi050.stdout:360 2024-04-02T16:25:31.017 INFO:teuthology.orchestra.run.smithi050.stdout:360.out 2024-04-02T16:25:31.017 INFO:teuthology.orchestra.run.smithi050.stdout:361 2024-04-02T16:25:31.017 INFO:teuthology.orchestra.run.smithi050.stdout:361.out 2024-04-02T16:25:31.017 INFO:teuthology.orchestra.run.smithi050.stdout:362 2024-04-02T16:25:31.017 INFO:teuthology.orchestra.run.smithi050.stdout:362.out 2024-04-02T16:25:31.017 INFO:teuthology.orchestra.run.smithi050.stdout:363 2024-04-02T16:25:31.018 INFO:teuthology.orchestra.run.smithi050.stdout:363.out 2024-04-02T16:25:31.018 INFO:teuthology.orchestra.run.smithi050.stdout:364 2024-04-02T16:25:31.018 INFO:teuthology.orchestra.run.smithi050.stdout:364.out 2024-04-02T16:25:31.018 INFO:teuthology.orchestra.run.smithi050.stdout:365 2024-04-02T16:25:31.018 INFO:teuthology.orchestra.run.smithi050.stdout:365.out 2024-04-02T16:25:31.018 INFO:teuthology.orchestra.run.smithi050.stdout:366 2024-04-02T16:25:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:366.out 2024-04-02T16:25:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:367 2024-04-02T16:25:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:367.out 2024-04-02T16:25:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:368 2024-04-02T16:25:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:368.out 2024-04-02T16:25:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:369 2024-04-02T16:25:31.019 INFO:teuthology.orchestra.run.smithi050.stdout:369.out 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:370 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:370.out 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:371 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:371.out 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:372 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:372.out 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:373 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:373.out 2024-04-02T16:25:31.020 INFO:teuthology.orchestra.run.smithi050.stdout:374 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:374.out 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:375 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:375.out 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:376 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:376.out 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:377 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:377.out 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:378 2024-04-02T16:25:31.021 INFO:teuthology.orchestra.run.smithi050.stdout:378.out 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:379 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:379.out 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:380 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:380.out 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:381 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:381.out 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:382 2024-04-02T16:25:31.022 INFO:teuthology.orchestra.run.smithi050.stdout:382.out 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:383 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:383.out 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:384 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:384.out 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:385 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:385.out 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:386 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:386.out 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:387 2024-04-02T16:25:31.023 INFO:teuthology.orchestra.run.smithi050.stdout:387.out 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:388 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:388.out 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:389 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:389.out 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:390 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:390.out 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:391 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:391.out 2024-04-02T16:25:31.024 INFO:teuthology.orchestra.run.smithi050.stdout:392 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:392.out 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:393 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:393.out 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:394 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:394.out 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:395 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:395.out 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:396 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:396.out 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:397 2024-04-02T16:25:31.025 INFO:teuthology.orchestra.run.smithi050.stdout:397.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:398 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:398.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:399 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:399.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:400 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:400.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:401 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:401.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:402 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:402.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:403 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:403.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:404 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:404.out 2024-04-02T16:25:31.026 INFO:teuthology.orchestra.run.smithi050.stdout:405 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:405.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:406 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:406.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:407 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:407.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:408 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:408.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:409 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:409.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:410 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:410.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:411 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:411.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:412 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:412.out 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:413 2024-04-02T16:25:31.027 INFO:teuthology.orchestra.run.smithi050.stdout:413.out 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:414 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:414.out 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:415 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:415.out 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:416 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:416.out 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:417 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:417.out 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:418 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:418.out 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:419 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:419.out 2024-04-02T16:25:31.028 INFO:teuthology.orchestra.run.smithi050.stdout:420 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:420.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:421 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:421.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:422 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:422.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:423 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:423.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:424 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:424.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:425 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:425.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:426 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:426.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:427 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:427.out 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:428 2024-04-02T16:25:31.029 INFO:teuthology.orchestra.run.smithi050.stdout:428.out 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:429 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:429.out 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:430 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:430.out 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:431 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:431.out 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:432 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:432.out 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:433 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:433.out 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:434 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:434.out 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:435 2024-04-02T16:25:31.030 INFO:teuthology.orchestra.run.smithi050.stdout:435.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:436 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:436.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:437 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:437.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:438 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:438.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:439 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:439.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:440 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:440.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:441 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:441.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:442 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:442.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:443 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:443.out 2024-04-02T16:25:31.031 INFO:teuthology.orchestra.run.smithi050.stdout:444 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:444.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:445 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:445.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:446 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:446.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:447 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:447.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:448 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:448.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:449 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:449.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:450 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:450.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:451 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:451.out 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:452 2024-04-02T16:25:31.032 INFO:teuthology.orchestra.run.smithi050.stdout:452.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:453 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:453.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:454 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:454.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:455 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:455.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:456 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:456.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:457 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:457.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:458 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:458.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:459 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:459.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:460 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:460.out 2024-04-02T16:25:31.033 INFO:teuthology.orchestra.run.smithi050.stdout:461 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:461.out 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:462 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:462.out 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:463 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:463.out 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:464 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:464.out 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:465 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:465.out 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:466 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:466.out 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:467 2024-04-02T16:25:31.034 INFO:teuthology.orchestra.run.smithi050.stdout:467.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:468 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:468.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:469 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:469.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:470 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:470.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:471 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:471.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:472 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:472.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:473 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:473.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:474 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:474.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:475 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:475.out 2024-04-02T16:25:31.035 INFO:teuthology.orchestra.run.smithi050.stdout:476 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:476.out 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:477 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:477.out 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:478 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:478.out 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:479 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:479.out 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:480 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:480.out 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:481 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:481.out 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:482 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:482.out 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:483 2024-04-02T16:25:31.036 INFO:teuthology.orchestra.run.smithi050.stdout:483.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:484 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:484.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:485 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:485.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:486 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:486.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:487 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:487.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:488 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:488.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:489 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:489.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:490 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:490.out 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:491 2024-04-02T16:25:31.037 INFO:teuthology.orchestra.run.smithi050.stdout:491.out 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:492 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:492.out 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:493 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:493.out 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:494 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:494.out 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:495 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:495.out 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:496 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:496.out 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:497 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:497.out 2024-04-02T16:25:31.038 INFO:teuthology.orchestra.run.smithi050.stdout:498 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:498.out 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:499 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:499.out 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:500 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:500.out 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:501 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:501.out 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:502 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:502.out 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:503 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:503.out 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:504 2024-04-02T16:25:31.039 INFO:teuthology.orchestra.run.smithi050.stdout:504.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:505 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:505.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:506 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:506.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:507 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:507.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:508 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:508.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:509 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:509.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:510 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:510.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:511 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:511.out 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:512 2024-04-02T16:25:31.040 INFO:teuthology.orchestra.run.smithi050.stdout:512.out 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:513 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:513.out 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:514 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:514.out 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:515 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:515.out 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:516 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:516.out 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:517 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:517.out 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:518 2024-04-02T16:25:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:518.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:519 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:519.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:520 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:520.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:521 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:521.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:522 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:522.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:523 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:523.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:524 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:524.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:525 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:525.out 2024-04-02T16:25:31.042 INFO:teuthology.orchestra.run.smithi050.stdout:526 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:526.out 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:527 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:527.out 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:528 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:528.out 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:529 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:529.out 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:530 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:530.out 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:531 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:531.out 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:532 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:532.out 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:533 2024-04-02T16:25:31.043 INFO:teuthology.orchestra.run.smithi050.stdout:533.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:534 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:534.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:535 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:535.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:536 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:536.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:537 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:537.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:538 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:538.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:539 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:539.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:540 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:540.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:541 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:541.out 2024-04-02T16:25:31.044 INFO:teuthology.orchestra.run.smithi050.stdout:542 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:542.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:543 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:543.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:544 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:544.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:545 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:545.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:546 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:546.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:547 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:547.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:548 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:548.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:549 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:549.out 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:550 2024-04-02T16:25:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:550.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:551 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:551.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:552 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:552.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:553 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:553.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:554 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:554.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:555 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:555.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:556 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:556.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:557 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:557.out 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:558 2024-04-02T16:25:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:558.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:559 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:559.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:560 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:560.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:561 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:561.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:562 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:562.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:563 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:563.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:564 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:564.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:565 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:565.out 2024-04-02T16:25:31.047 INFO:teuthology.orchestra.run.smithi050.stdout:566 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:566.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:567 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:567.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:568 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:568.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:569 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:569.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:570 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:570.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:571 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:571.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:572 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:572.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:573 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:573.out 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:574 2024-04-02T16:25:31.048 INFO:teuthology.orchestra.run.smithi050.stdout:574.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:575 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:575.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:576 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:576.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:577 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:577.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:578 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:578.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:579 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:579.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:580 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:580.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:581 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:581.out 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:582 2024-04-02T16:25:31.049 INFO:teuthology.orchestra.run.smithi050.stdout:582.out 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:583 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:583.out 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:584 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:584.out 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:585 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:585.out 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:586 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:586.out 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:587 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:587.out 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:588 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:588.out 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:589 2024-04-02T16:25:31.050 INFO:teuthology.orchestra.run.smithi050.stdout:589.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:590 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:590.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:591 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:591.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:592 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:592.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:593 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:593.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:594 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:594.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:595 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:595.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:596 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:596.out 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:597 2024-04-02T16:25:31.051 INFO:teuthology.orchestra.run.smithi050.stdout:597.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:598 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:598.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:599 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:599.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:600 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:600.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:601 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:601.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:602 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:602.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:603 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:603.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:604 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:604.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:605 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:605.out 2024-04-02T16:25:31.052 INFO:teuthology.orchestra.run.smithi050.stdout:606 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:606.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:607 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:607.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:608 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:608.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:609 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:609.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:610 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:610.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:611 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:611.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:612 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:612.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:613 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:613.out 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:614 2024-04-02T16:25:31.053 INFO:teuthology.orchestra.run.smithi050.stdout:614.out 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:615 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:615.out 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:616 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:616.out 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:617 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:617.out 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:618 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:618.out 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:619 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:619.out 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:620 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:620.out 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:621 2024-04-02T16:25:31.054 INFO:teuthology.orchestra.run.smithi050.stdout:621.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:622 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:622.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:623 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:623.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:624 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:624.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:625 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:625.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:626 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:626.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:627 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:627.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:628 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:628.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:629 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:629.out 2024-04-02T16:25:31.055 INFO:teuthology.orchestra.run.smithi050.stdout:630 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:630.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:631 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:631.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:632 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:632.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:633 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:633.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:634 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:634.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:635 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:635.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:636 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:636.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:637 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:637.out 2024-04-02T16:25:31.056 INFO:teuthology.orchestra.run.smithi050.stdout:638 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:638.out 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:639 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:639.out 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:640 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:640.out 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:641 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:641.out 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:642 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:642.out 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:643 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:643.out 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:644 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:644.out 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:645 2024-04-02T16:25:31.057 INFO:teuthology.orchestra.run.smithi050.stdout:645.out 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:646 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:646.out 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:647 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:647.out 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:648 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:648.out 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:649 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:649.out 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:650 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:650.out 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:651 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:651.out 2024-04-02T16:25:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:652 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:652.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:653 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:653.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:654 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:654.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:655 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:655.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:656 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:656.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:657 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:657.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:658 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:658.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:659 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:659.out 2024-04-02T16:25:31.059 INFO:teuthology.orchestra.run.smithi050.stdout:660 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:660.out 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:661 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:661.out 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:662 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:662.out 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:663 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:663.out 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:664 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:664.out 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:665 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:665.out 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:666 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:666.out 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:667 2024-04-02T16:25:31.060 INFO:teuthology.orchestra.run.smithi050.stdout:667.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:668 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:668.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:669 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:669.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:670 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:670.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:671 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:671.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:672 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:672.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:673 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:673.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:674 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:674.out 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:675 2024-04-02T16:25:31.061 INFO:teuthology.orchestra.run.smithi050.stdout:675.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:676 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:676.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:677 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:677.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:678 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:678.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:679 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:679.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:680 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:680.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:681 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:681.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:682 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:682.out 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:683 2024-04-02T16:25:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:683.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:684 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:684.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:685 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:685.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:686 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:686.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:687 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:687.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:688 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:688.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:689 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:689.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:690 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:690.out 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:691 2024-04-02T16:25:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:691.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:692 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:692.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:693 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:693.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:694 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:694.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:695 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:695.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:696 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:696.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:697 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:697.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:698 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:698.out 2024-04-02T16:25:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:699 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:699.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:700 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:700.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:701 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:701.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:702 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:702.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:703 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:703.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:704 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:704.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:705 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:705.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:706 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:706.out 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:707 2024-04-02T16:25:31.065 INFO:teuthology.orchestra.run.smithi050.stdout:707.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:708 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:708.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:709 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:709.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:710 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:710.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:711 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:711.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:712 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:712.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:713 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:713.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:714 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:714.out 2024-04-02T16:25:31.066 INFO:teuthology.orchestra.run.smithi050.stdout:715 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:715.out 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:716 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:716.out 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:717 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:717.out 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:718 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:718.out 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:719 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:719.out 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:720 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:720.out 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:721 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:721.out 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:722 2024-04-02T16:25:31.067 INFO:teuthology.orchestra.run.smithi050.stdout:722.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:723 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:723.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:724 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:724.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:725 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:725.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:726 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:726.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:727 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:727.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:728 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:728.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:729 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:729.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:730 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:730.out 2024-04-02T16:25:31.068 INFO:teuthology.orchestra.run.smithi050.stdout:731 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:731.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:732 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:732.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:733 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:733.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:734 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:734.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:735 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:735.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:736 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:736.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:737 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:737.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:738 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:738.out 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:739 2024-04-02T16:25:31.069 INFO:teuthology.orchestra.run.smithi050.stdout:739.out 2024-04-02T16:25:31.070 INFO:teuthology.orchestra.run.smithi050.stdout:group.list 2024-04-02T16:25:31.070 INFO:teuthology.orchestra.run.smithi050.stdout:Makefile 2024-04-02T16:25:31.070 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-04-02T16:25:31.246 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:25:31.250 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:25:31.250 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:25:31.251 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:25:31.251 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:33.043 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:25:33.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:25:36.406 INFO:teuthology.orchestra.run.smithi050.stdout:generic/001 5s 2024-04-02T16:25:36.410 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/001 2024-04-02T16:25:36.411 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:25:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:36.710 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:25:36.710 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-04-02T16:25:37.012 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:25:37.015 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:25:37.016 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:25:37.016 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:25:37.017 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:37.361 INFO:teuthology.orchestra.run.smithi050.stdout:generic/002 0s 2024-04-02T16:25:37.366 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/002 2024-04-02T16:25:37.366 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:25:37.368 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:37.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:25:37.404 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-04-02T16:25:37.597 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:25:37.600 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:25:37.601 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:25:37.601 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:25:37.602 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:37.800 INFO:teuthology.orchestra.run.smithi050.stdout:generic/003 [not run] atime not maintained by ceph 2024-04-02T16:25:37.804 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/003 2024-04-02T16:25:37.804 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/003 2024-04-02T16:25:37.804 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:25:37.806 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:37.817 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:25:37.817 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-04-02T16:25:37.983 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:25:37.986 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:25:37.987 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:25:37.987 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:25:37.987 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:38.187 INFO:teuthology.orchestra.run.smithi050.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-04-02T16:25:38.190 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/004 2024-04-02T16:25:38.190 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/004 2024-04-02T16:25:38.190 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:25:38.192 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:38.202 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:25:38.203 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-04-02T16:25:38.380 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:25:38.383 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:25:38.383 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:25:38.384 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:25:38.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:38.723 INFO:teuthology.orchestra.run.smithi050.stdout:generic/005 0s 2024-04-02T16:25:38.728 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/005 2024-04-02T16:25:38.729 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:25:38.731 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:25:38.775 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:25:38.775 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-04-02T16:25:38.952 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:25:38.956 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:25:38.956 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:25:38.957 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:25:38.958 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:03.104 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:26:03.107 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:26:04.442 INFO:teuthology.orchestra.run.smithi050.stdout:generic/006 25s 2024-04-02T16:26:04.459 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/006 2024-04-02T16:26:04.459 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:26:04.463 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:04.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:26:04.514 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-04-02T16:26:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:26:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:26:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:26:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:26:04.732 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:33.267 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:26:33.270 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:26:55.646 INFO:teuthology.orchestra.run.smithi050.stdout:generic/007 51s 2024-04-02T16:26:55.722 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/007 2024-04-02T16:26:55.722 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:26:55.725 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:55.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:26:55.798 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-04-02T16:26:55.986 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:26:55.990 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:26:55.990 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:26:55.991 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:26:55.991 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:56.234 INFO:teuthology.orchestra.run.smithi050.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-02T16:26:56.238 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/008 2024-04-02T16:26:56.238 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/008 2024-04-02T16:26:56.238 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:26:56.240 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:56.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:26:56.253 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-04-02T16:26:56.437 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:26:56.441 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:26:56.441 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:26:56.442 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:26:56.442 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:56.718 INFO:teuthology.orchestra.run.smithi050.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-02T16:26:56.722 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/009 2024-04-02T16:26:56.722 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/009 2024-04-02T16:26:56.722 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:26:56.724 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:56.738 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:26:56.738 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-04-02T16:26:56.918 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:26:56.922 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:26:56.922 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:26:56.923 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:26:56.923 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:57.444 INFO:teuthology.orchestra.run.smithi050.stdout:generic/010 0s 2024-04-02T16:26:57.517 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/010 2024-04-02T16:26:57.517 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:26:57.519 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:26:57.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:26:57.551 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-04-02T16:26:57.736 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:26:57.740 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:26:57.740 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:26:57.741 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:26:57.741 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:27:03.577 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:27:03.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:27:26.226 INFO:teuthology.orchestra.run.smithi050.stdout:generic/011 29s 2024-04-02T16:27:26.231 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/011 2024-04-02T16:27:26.231 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:27:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:27:26.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:27:26.272 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-04-02T16:27:26.461 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:27:26.465 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:27:26.465 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:27:26.466 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:27:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:27:26.741 INFO:teuthology.orchestra.run.smithi050.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:27:26.745 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/012 2024-04-02T16:27:26.745 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/012 2024-04-02T16:27:26.745 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:27:26.747 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:27:26.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:27:26.791 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-04-02T16:27:26.997 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:27:27.001 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:27:27.001 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:27:27.002 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:27:27.002 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:27:33.845 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:27:33.848 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:28:03.905 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:28:03.908 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:28:30.787 INFO:teuthology.orchestra.run.smithi050.stdout:generic/013 63s 2024-04-02T16:28:30.792 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/013 2024-04-02T16:28:30.793 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:30.795 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:30.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:30.820 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-04-02T16:28:31.058 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:31.062 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:31.063 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:31.064 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:31.064 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:31.356 INFO:teuthology.orchestra.run.smithi050.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-04-02T16:28:31.362 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/014 2024-04-02T16:28:31.362 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/014 2024-04-02T16:28:31.362 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:31.365 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:31.386 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:31.386 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-04-02T16:28:31.614 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:31.619 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:31.620 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:31.621 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:31.621 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:31.884 INFO:teuthology.orchestra.run.smithi050.stdout:generic/015 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T16:28:31.888 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/015 2024-04-02T16:28:31.889 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/015 2024-04-02T16:28:31.889 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:31.891 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:31.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:31.907 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-04-02T16:28:32.135 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:32.140 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:32.140 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:32.141 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:32.141 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:32.497 INFO:teuthology.orchestra.run.smithi050.stdout:generic/016 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:28:32.501 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/016 2024-04-02T16:28:32.501 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/016 2024-04-02T16:28:32.502 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:32.504 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:32.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:32.540 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-04-02T16:28:32.815 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:32.821 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:32.821 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:32.822 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:32.822 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:33.094 INFO:teuthology.orchestra.run.smithi050.stdout:generic/017 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-02T16:28:33.099 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/017 2024-04-02T16:28:33.099 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/017 2024-04-02T16:28:33.099 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:33.102 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:33.119 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:33.120 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-04-02T16:28:33.361 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:33.365 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:33.366 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:33.367 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:33.367 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:33.699 INFO:teuthology.orchestra.run.smithi050.stdout:generic/018 [not run] defragmentation not supported for fstype "ceph" 2024-04-02T16:28:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/018 2024-04-02T16:28:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/018 2024-04-02T16:28:33.707 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:33.710 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:33.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:33.725 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-04-02T16:28:33.959 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:33.963 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:33.963 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:33.964 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:33.965 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:34.225 INFO:teuthology.orchestra.run.smithi050.stdout:generic/019 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:28:34.229 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/019 2024-04-02T16:28:34.229 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/019 2024-04-02T16:28:34.229 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:34.231 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:34.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:34.246 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-04-02T16:28:34.485 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:34.490 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:34.491 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:34.492 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:34.492 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:34.557 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:28:34.559 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:28:38.148 INFO:teuthology.orchestra.run.smithi050.stdout:generic/020 4s 2024-04-02T16:28:38.216 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/020 2024-04-02T16:28:38.216 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:38.219 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:38.256 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:38.256 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-04-02T16:28:38.553 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:38.558 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:38.558 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:38.559 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:38.560 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:38.918 INFO:teuthology.orchestra.run.smithi050.stdout:generic/021 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:28:38.924 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/021 2024-04-02T16:28:38.924 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/021 2024-04-02T16:28:38.924 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:38.927 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:38.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:38.961 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-04-02T16:28:39.246 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:39.252 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:39.252 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:39.253 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:39.253 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:39.600 INFO:teuthology.orchestra.run.smithi050.stdout:generic/022 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:28:39.605 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/022 2024-04-02T16:28:39.605 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/022 2024-04-02T16:28:39.605 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:39.609 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:39.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:39.633 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-04-02T16:28:39.907 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:39.912 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:39.913 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:39.914 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:39.914 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:41.041 INFO:teuthology.orchestra.run.smithi050.stdout:generic/023 1s 2024-04-02T16:28:41.074 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/023 2024-04-02T16:28:41.075 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:41.077 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:41.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:41.186 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-04-02T16:28:41.440 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:41.445 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:41.446 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:41.447 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:41.447 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:41.735 INFO:teuthology.orchestra.run.smithi050.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-04-02T16:28:41.740 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/024 2024-04-02T16:28:41.740 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/024 2024-04-02T16:28:41.741 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:41.744 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:41.775 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:41.775 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-04-02T16:28:42.067 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:42.073 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:42.073 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:42.075 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:42.075 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:42.394 INFO:teuthology.orchestra.run.smithi050.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-04-02T16:28:42.399 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/025 2024-04-02T16:28:42.399 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/025 2024-04-02T16:28:42.400 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:42.402 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:42.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:42.448 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-04-02T16:28:42.726 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:42.729 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:42.729 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:42.730 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:42.731 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:43.030 INFO:teuthology.orchestra.run.smithi050.stdout:generic/026 [not run] ceph does not define maximum ACL count 2024-04-02T16:28:43.038 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/026 2024-04-02T16:28:43.038 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/026 2024-04-02T16:28:43.038 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:43.039 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:43.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:43.057 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-04-02T16:28:43.301 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:43.306 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:43.307 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:43.308 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:43.308 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:43.574 INFO:teuthology.orchestra.run.smithi050.stdout:generic/027 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T16:28:43.579 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/027 2024-04-02T16:28:43.580 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/027 2024-04-02T16:28:43.580 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:43.582 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:43.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:43.601 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-04-02T16:28:43.846 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:43.851 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:43.851 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:43.853 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:43.853 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:49.151 INFO:teuthology.orchestra.run.smithi050.stdout:generic/028 6s 2024-04-02T16:28:49.159 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/028 2024-04-02T16:28:49.159 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:49.162 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:49.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:49.181 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-04-02T16:28:49.414 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:49.418 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:49.418 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:49.420 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:49.420 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:53.207 INFO:teuthology.orchestra.run.smithi050.stdout:generic/029 4s 2024-04-02T16:28:53.258 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/029 2024-04-02T16:28:53.258 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:53.261 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:53.272 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:53.273 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-04-02T16:28:53.513 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:53.518 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:53.519 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:53.519 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:53.520 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:54.939 INFO:teuthology.orchestra.run.smithi050.stdout:generic/030 1s 2024-04-02T16:28:54.946 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/030 2024-04-02T16:28:54.946 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:54.949 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:54.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:54.962 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-04-02T16:28:55.193 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:55.197 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:55.198 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:55.199 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:55.199 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:55.544 INFO:teuthology.orchestra.run.smithi050.stdout:generic/031 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-02T16:28:55.548 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/031 2024-04-02T16:28:55.549 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/031 2024-04-02T16:28:55.549 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:55.553 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:55.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:55.572 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-04-02T16:28:55.805 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:55.809 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:55.809 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:55.811 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:55.811 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:56.084 INFO:teuthology.orchestra.run.smithi050.stdout:generic/032 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:28:56.089 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/032 2024-04-02T16:28:56.089 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/032 2024-04-02T16:28:56.089 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:56.092 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:56.115 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:56.115 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-04-02T16:28:56.352 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:56.358 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:56.358 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:56.359 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:56.359 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:56.703 INFO:teuthology.orchestra.run.smithi050.stdout:generic/033 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-02T16:28:56.708 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/033 2024-04-02T16:28:56.708 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/033 2024-04-02T16:28:56.708 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:56.711 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:56.734 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:56.734 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-04-02T16:28:56.970 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:56.975 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:56.976 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:56.977 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:56.977 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:57.250 INFO:teuthology.orchestra.run.smithi050.stdout:generic/034 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:28:57.259 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/034 2024-04-02T16:28:57.259 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/034 2024-04-02T16:28:57.259 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:57.259 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:57.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:57.275 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-04-02T16:28:57.501 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:57.506 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:57.506 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:57.508 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:57.508 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:57.833 INFO:teuthology.orchestra.run.smithi050.stdout:generic/035 0s 2024-04-02T16:28:57.839 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/035 2024-04-02T16:28:57.839 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:28:57.844 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:28:57.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:28:57.871 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-04-02T16:28:58.169 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:28:58.173 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:28:58.173 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:28:58.174 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:28:58.175 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:04.845 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:29:04.847 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:29:08.532 INFO:teuthology.orchestra.run.smithi050.stdout:generic/036 10s 2024-04-02T16:29:08.706 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/036 2024-04-02T16:29:08.706 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:08.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:08.710 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-04-02T16:29:08.947 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:08.952 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:08.952 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:08.953 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:13.454 INFO:teuthology.orchestra.run.smithi050.stdout:generic/037 4s 2024-04-02T16:29:13.460 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/037 2024-04-02T16:29:13.461 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:13.463 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:13.473 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:13.474 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-04-02T16:29:13.713 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:13.718 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:13.720 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:13.720 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:13.720 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:14.007 INFO:teuthology.orchestra.run.smithi050.stdout:generic/038 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:29:14.012 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/038 2024-04-02T16:29:14.013 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/038 2024-04-02T16:29:14.013 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:14.017 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:14.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:14.034 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-04-02T16:29:14.278 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:14.283 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:14.284 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:14.285 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:14.285 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:14.568 INFO:teuthology.orchestra.run.smithi050.stdout:generic/039 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:14.572 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/039 2024-04-02T16:29:14.572 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/039 2024-04-02T16:29:14.572 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:14.574 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:14.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:14.592 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-04-02T16:29:14.830 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:14.835 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:14.836 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:14.837 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:14.837 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:15.125 INFO:teuthology.orchestra.run.smithi050.stdout:generic/040 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:15.131 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/040 2024-04-02T16:29:15.131 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/040 2024-04-02T16:29:15.131 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:15.133 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:15.152 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:15.152 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-04-02T16:29:15.394 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:15.398 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:15.399 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:15.400 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:15.401 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:15.696 INFO:teuthology.orchestra.run.smithi050.stdout:generic/041 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:15.700 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/041 2024-04-02T16:29:15.700 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/041 2024-04-02T16:29:15.701 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:15.704 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:15.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:15.722 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-04-02T16:29:15.967 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:15.972 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:15.972 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:15.974 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:15.974 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:16.319 INFO:teuthology.orchestra.run.smithi050.stdout:generic/042 [not run] ceph does not support shutdown 2024-04-02T16:29:16.329 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/042 2024-04-02T16:29:16.329 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/042 2024-04-02T16:29:16.329 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:16.332 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:16.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:16.346 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-04-02T16:29:16.588 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:16.596 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:16.597 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:16.598 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:16.598 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:16.931 INFO:teuthology.orchestra.run.smithi050.stdout:generic/043 [not run] ceph does not support shutdown 2024-04-02T16:29:16.941 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/043 2024-04-02T16:29:16.941 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/043 2024-04-02T16:29:16.941 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:16.944 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:16.958 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:16.958 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-04-02T16:29:17.197 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:17.202 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:17.203 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:17.204 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:17.204 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:17.557 INFO:teuthology.orchestra.run.smithi050.stdout:generic/044 [not run] ceph does not support shutdown 2024-04-02T16:29:17.566 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/044 2024-04-02T16:29:17.566 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/044 2024-04-02T16:29:17.566 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:17.569 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:17.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:17.582 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-04-02T16:29:17.823 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:17.829 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:17.830 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:17.831 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:17.831 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:18.163 INFO:teuthology.orchestra.run.smithi050.stdout:generic/045 [not run] ceph does not support shutdown 2024-04-02T16:29:18.172 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/045 2024-04-02T16:29:18.172 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/045 2024-04-02T16:29:18.172 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:18.176 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:18.191 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:18.191 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-04-02T16:29:18.440 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:18.444 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:18.445 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:18.446 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:18.446 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:18.792 INFO:teuthology.orchestra.run.smithi050.stdout:generic/046 [not run] ceph does not support shutdown 2024-04-02T16:29:18.801 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/046 2024-04-02T16:29:18.801 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/046 2024-04-02T16:29:18.801 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:18.804 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:18.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:18.819 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-04-02T16:29:19.051 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:19.056 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:19.057 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:19.058 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:19.400 INFO:teuthology.orchestra.run.smithi050.stdout:generic/047 [not run] ceph does not support shutdown 2024-04-02T16:29:19.418 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/047 2024-04-02T16:29:19.419 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/047 2024-04-02T16:29:19.419 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:19.421 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:19.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:19.439 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-04-02T16:29:19.672 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:19.677 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:19.678 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:19.679 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:19.679 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:20.031 INFO:teuthology.orchestra.run.smithi050.stdout:generic/048 [not run] ceph does not support shutdown 2024-04-02T16:29:20.039 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/048 2024-04-02T16:29:20.039 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/048 2024-04-02T16:29:20.039 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:20.041 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:20.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:20.056 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-04-02T16:29:20.289 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:20.294 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:20.295 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:20.296 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:20.296 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:20.634 INFO:teuthology.orchestra.run.smithi050.stdout:generic/049 [not run] ceph does not support shutdown 2024-04-02T16:29:20.642 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/049 2024-04-02T16:29:20.642 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/049 2024-04-02T16:29:20.642 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:20.644 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:20.660 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:20.660 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-04-02T16:29:20.903 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:20.908 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:20.909 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:20.910 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:20.910 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:21.245 INFO:teuthology.orchestra.run.smithi050.stdout:generic/050 [not run] ceph does not support shutdown 2024-04-02T16:29:21.262 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/050 2024-04-02T16:29:21.262 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/050 2024-04-02T16:29:21.262 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:21.265 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:21.279 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:21.279 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-04-02T16:29:21.517 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:21.522 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:21.523 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:21.524 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:21.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:21.868 INFO:teuthology.orchestra.run.smithi050.stdout:generic/051 [not run] ceph does not support shutdown 2024-04-02T16:29:21.873 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/051 2024-04-02T16:29:21.873 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/051 2024-04-02T16:29:21.873 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:21.876 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:21.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:21.889 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-04-02T16:29:22.121 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:22.126 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:22.127 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:22.128 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:22.128 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:22.471 INFO:teuthology.orchestra.run.smithi050.stdout:generic/052 [not run] ceph does not support shutdown 2024-04-02T16:29:22.492 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/052 2024-04-02T16:29:22.492 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/052 2024-04-02T16:29:22.492 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:22.495 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:22.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:22.512 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-04-02T16:29:22.760 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:22.765 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:22.765 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:22.766 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:22.766 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:23.412 INFO:teuthology.orchestra.run.smithi050.stdout:generic/053 1s 2024-04-02T16:29:23.456 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/053 2024-04-02T16:29:23.456 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:23.459 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:23.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:23.468 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-04-02T16:29:23.703 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:23.708 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:23.709 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:23.710 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:23.710 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:24.074 INFO:teuthology.orchestra.run.smithi050.stdout:generic/054 [not run] ceph does not support shutdown 2024-04-02T16:29:24.082 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/054 2024-04-02T16:29:24.082 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/054 2024-04-02T16:29:24.082 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:24.085 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:24.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:24.107 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-04-02T16:29:24.345 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:24.350 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:24.350 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:24.352 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:24.352 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:24.701 INFO:teuthology.orchestra.run.smithi050.stdout:generic/055 [not run] ceph does not support shutdown 2024-04-02T16:29:24.710 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/055 2024-04-02T16:29:24.710 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/055 2024-04-02T16:29:24.710 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:24.714 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:24.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:24.732 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-04-02T16:29:24.976 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:24.981 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:24.982 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:24.984 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:24.984 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:25.288 INFO:teuthology.orchestra.run.smithi050.stdout:generic/056 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:25.293 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/056 2024-04-02T16:29:25.294 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/056 2024-04-02T16:29:25.294 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:25.296 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:25.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:25.319 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-04-02T16:29:25.567 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:25.572 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:25.573 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:25.574 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:25.574 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:25.876 INFO:teuthology.orchestra.run.smithi050.stdout:generic/057 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:25.881 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/057 2024-04-02T16:29:25.881 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/057 2024-04-02T16:29:25.881 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:25.884 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:25.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:25.905 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-04-02T16:29:26.138 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:26.144 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:26.145 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:26.146 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:26.146 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:26.517 INFO:teuthology.orchestra.run.smithi050.stdout:generic/058 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:29:26.522 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/058 2024-04-02T16:29:26.522 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/058 2024-04-02T16:29:26.522 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:26.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:26.562 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:26.562 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-04-02T16:29:26.877 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:26.885 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:26.885 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:26.886 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:26.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:27.160 INFO:teuthology.orchestra.run.smithi050.stdout:generic/059 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:27.165 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/059 2024-04-02T16:29:27.165 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/059 2024-04-02T16:29:27.165 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:27.168 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:27.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:27.183 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-04-02T16:29:27.415 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:27.421 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:27.422 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:27.423 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:27.423 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:27.791 INFO:teuthology.orchestra.run.smithi050.stdout:generic/060 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:29:27.796 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/060 2024-04-02T16:29:27.796 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/060 2024-04-02T16:29:27.796 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:27.799 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:27.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:27.831 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-04-02T16:29:28.157 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:28.163 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:28.163 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:28.164 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:28.164 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:28.533 INFO:teuthology.orchestra.run.smithi050.stdout:generic/061 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:29:28.540 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/061 2024-04-02T16:29:28.540 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/061 2024-04-02T16:29:28.540 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:28.542 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:28.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:28.573 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-04-02T16:29:28.890 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:28.897 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:28.897 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:30.530 INFO:teuthology.orchestra.run.smithi050.stdout:generic/062 1s 2024-04-02T16:29:30.538 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/062 2024-04-02T16:29:30.538 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:30.540 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:30.554 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:30.554 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-04-02T16:29:30.832 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:30.836 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:30.837 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:30.838 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:31.276 INFO:teuthology.orchestra.run.smithi050.stdout:generic/063 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:29:31.281 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/063 2024-04-02T16:29:31.281 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/063 2024-04-02T16:29:31.281 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:31.285 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:31.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:31.338 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-04-02T16:29:31.642 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:31.647 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:31.647 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:31.649 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:31.649 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:31.943 INFO:teuthology.orchestra.run.smithi050.stdout:generic/064 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-02T16:29:31.948 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/064 2024-04-02T16:29:31.948 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/064 2024-04-02T16:29:31.948 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:31.951 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:31.970 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:31.970 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-04-02T16:29:32.206 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:32.211 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:32.212 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:32.213 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:32.213 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:32.510 INFO:teuthology.orchestra.run.smithi050.stdout:generic/065 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:32.515 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/065 2024-04-02T16:29:32.515 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/065 2024-04-02T16:29:32.515 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:32.518 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:32.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:32.536 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-04-02T16:29:32.779 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:32.784 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:32.784 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:32.785 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:32.786 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:33.061 INFO:teuthology.orchestra.run.smithi050.stdout:generic/066 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:33.065 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/066 2024-04-02T16:29:33.066 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/066 2024-04-02T16:29:33.066 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:33.068 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:33.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:33.085 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-04-02T16:29:33.337 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:33.341 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:33.342 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:33.343 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:33.343 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:33.630 INFO:teuthology.orchestra.run.smithi050.stdout:generic/067 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:29:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/067 2024-04-02T16:29:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/067 2024-04-02T16:29:33.636 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:33.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:33.662 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-04-02T16:29:33.897 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:33.901 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:33.901 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:33.903 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:33.903 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:34.177 INFO:teuthology.orchestra.run.smithi050.stdout:generic/068 [not run] ceph does not support freezing 2024-04-02T16:29:34.181 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/068 2024-04-02T16:29:34.181 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/068 2024-04-02T16:29:34.182 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:34.184 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:34.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:34.199 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-04-02T16:29:34.436 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:34.442 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:34.442 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:34.442 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:34.442 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:34.921 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:29:34.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:29:41.306 INFO:teuthology.orchestra.run.smithi050.stdout:generic/069 7s 2024-04-02T16:29:41.316 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/069 2024-04-02T16:29:41.316 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:29:41.318 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:29:41.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:29:41.333 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-04-02T16:29:41.611 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:29:41.616 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:29:41.617 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:29:41.618 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:29:41.618 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:05.013 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:30:05.016 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:30:05.078 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T16:30:05.078+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 159278) UID: 0 2024-04-02T16:30:05.079 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T16:30:05.080+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 159278) UID: 0 2024-04-02T16:30:06.162 INFO:teuthology.orchestra.run.smithi050.stdout:generic/070 25s 2024-04-02T16:30:06.168 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/070 2024-04-02T16:30:06.169 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:06.174 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:06.256 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:06.256 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-04-02T16:30:06.572 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:06.576 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:06.576 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:06.577 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:06.577 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:06.873 INFO:teuthology.orchestra.run.smithi050.stdout:generic/071 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T16:30:06.878 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/071 2024-04-02T16:30:06.878 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/071 2024-04-02T16:30:06.878 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:06.881 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:06.902 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:06.903 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-04-02T16:30:07.167 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:07.172 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:07.173 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:07.174 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:07.174 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:07.541 INFO:teuthology.orchestra.run.smithi050.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-02T16:30:07.545 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/072 2024-04-02T16:30:07.545 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/072 2024-04-02T16:30:07.545 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:07.549 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:07.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:07.570 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-04-02T16:30:07.806 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:07.812 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:07.812 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:07.813 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:08.102 INFO:teuthology.orchestra.run.smithi050.stdout:generic/073 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:30:08.106 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/073 2024-04-02T16:30:08.106 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/073 2024-04-02T16:30:08.106 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:08.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:08.125 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-04-02T16:30:08.378 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:08.383 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:08.384 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:08.385 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:23.979 INFO:teuthology.orchestra.run.smithi050.stdout:generic/074 15s 2024-04-02T16:30:24.057 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/074 2024-04-02T16:30:24.057 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:24.063 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:25.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:25.251 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-04-02T16:30:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:25.597 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:25.598 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:25.601 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:25.601 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:45.917 INFO:teuthology.orchestra.run.smithi050.stdout:generic/075 20s 2024-04-02T16:30:45.924 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/075 2024-04-02T16:30:45.924 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:45.929 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:45.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:45.992 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-04-02T16:30:46.330 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:46.336 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:46.337 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:46.338 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:46.339 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:46.632 INFO:teuthology.orchestra.run.smithi050.stdout:generic/076 [not run] require 172.21.15.50:6789:/scratch to be local device 2024-04-02T16:30:46.637 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/076 2024-04-02T16:30:46.638 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/076 2024-04-02T16:30:46.638 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:46.642 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:46.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:46.656 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-04-02T16:30:46.956 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:46.961 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:46.962 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:46.963 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:46.963 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:54.017 INFO:teuthology.orchestra.run.smithi050.stdout:generic/077 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T16:30:54.022 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/077 2024-04-02T16:30:54.022 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/077 2024-04-02T16:30:54.022 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:54.025 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:54.075 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:54.075 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-04-02T16:30:54.451 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:54.457 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:54.457 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:54.458 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:54.459 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:54.761 INFO:teuthology.orchestra.run.smithi050.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-04-02T16:30:54.766 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/078 2024-04-02T16:30:54.767 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/078 2024-04-02T16:30:54.767 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:54.769 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:54.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:54.807 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-04-02T16:30:55.168 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:55.175 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:55.175 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:55.176 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:55.177 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:55.489 INFO:teuthology.orchestra.run.smithi050.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-04-02T16:30:55.494 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/079 2024-04-02T16:30:55.494 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/079 2024-04-02T16:30:55.495 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:55.497 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:55.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:55.525 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-04-02T16:30:55.785 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:55.792 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:55.793 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:55.794 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:55.794 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:58.153 INFO:teuthology.orchestra.run.smithi050.stdout:generic/080 3s 2024-04-02T16:30:58.162 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/080 2024-04-02T16:30:58.162 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:58.165 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:58.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:58.185 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-04-02T16:30:58.436 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:58.443 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:58.443 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:58.445 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:58.445 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:58.798 INFO:teuthology.orchestra.run.smithi050.stdout:generic/081 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:30:58.802 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/081 2024-04-02T16:30:58.802 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/081 2024-04-02T16:30:58.803 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:58.806 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:58.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:58.819 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-04-02T16:30:59.077 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:59.083 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:59.083 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:59.085 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:59.085 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:59.374 INFO:teuthology.orchestra.run.smithi050.stdout:generic/082 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T16:30:59.380 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/082 2024-04-02T16:30:59.380 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/082 2024-04-02T16:30:59.380 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:59.384 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:59.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:59.400 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-04-02T16:30:59.652 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:30:59.657 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:30:59.657 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:30:59.658 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:30:59.658 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:59.944 INFO:teuthology.orchestra.run.smithi050.stdout:generic/083 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T16:30:59.948 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/083 2024-04-02T16:30:59.948 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/083 2024-04-02T16:30:59.948 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:30:59.951 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:30:59.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:30:59.967 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-04-02T16:31:00.226 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:31:00.231 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:31:00.231 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:31:00.232 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:31:00.232 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:05.983 INFO:teuthology.orchestra.run.smithi050.stdout:generic/084 5s 2024-04-02T16:31:05.992 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/084 2024-04-02T16:31:05.992 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:31:05.997 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:06.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:31:06.013 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-04-02T16:31:06.325 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:31:06.329 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:31:06.329 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:31:06.331 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:31:06.331 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:06.656 INFO:teuthology.orchestra.run.smithi050.stdout:generic/085 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:31:06.662 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/085 2024-04-02T16:31:06.662 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/085 2024-04-02T16:31:06.662 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:31:06.665 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:06.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:31:06.683 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-04-02T16:31:06.941 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:31:06.946 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:31:06.946 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:31:06.948 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:31:06.948 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:07.234 INFO:teuthology.orchestra.run.smithi050.stdout:generic/086 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:31:07.239 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/086 2024-04-02T16:31:07.240 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/086 2024-04-02T16:31:07.240 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:31:07.244 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:07.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:31:07.261 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-04-02T16:31:07.516 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:31:07.522 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:31:07.523 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:31:07.524 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:31:07.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:07.894 INFO:teuthology.orchestra.run.smithi050.stdout:generic/087 0s 2024-04-02T16:31:07.906 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/087 2024-04-02T16:31:07.906 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:31:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:07.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:31:07.977 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-04-02T16:31:08.356 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:31:08.362 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:31:08.363 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:31:08.364 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:31:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:08.689 INFO:teuthology.orchestra.run.smithi050.stdout:generic/088 0s 2024-04-02T16:31:08.697 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/088 2024-04-02T16:31:08.697 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:31:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:31:08.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:31:08.747 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-04-02T16:31:09.124 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:31:09.130 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:31:09.130 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:31:09.133 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:31:09.133 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:33:13.363 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:33:13.366 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:33:43.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:33:43.580 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:34:13.656 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:34:13.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:34:43.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:34:43.919 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:35:13.969 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:35:13.972 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:35:14.046 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T16:35:14.045+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 171886) UID: 0 2024-04-02T16:35:14.046 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T16:35:14.045+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 171886) UID: 0 2024-04-02T16:38:27.703 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:38:27.707 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:38:27.764 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T16:38:27.763+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173921) UID: 0 2024-04-02T16:38:27.764 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T16:38:27.764+0000 7fcf2c643640 -1 Fail to open '/proc/173921/cmdline' error = (2) No such file or directory 2024-04-02T16:38:27.764 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T16:38:27.764+0000 7fcf2c643640 -1 received signal: Hangup from (PID: 173921) UID: 0 2024-04-02T16:39:28.979 INFO:teuthology.orchestra.run.smithi050.stdout:generic/089 499s 2024-04-02T16:39:29.013 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/089 2024-04-02T16:39:29.013 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:39:29.025 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:39:29.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:39:29.585 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-04-02T16:39:30.029 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:39:30.033 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:39:30.033 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:39:30.034 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:39:30.034 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:39:30.277 INFO:teuthology.orchestra.run.smithi050.stdout:generic/090 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:39:30.281 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/090 2024-04-02T16:39:30.281 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/090 2024-04-02T16:39:30.281 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:39:30.283 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:39:30.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:39:30.298 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-04-02T16:39:30.738 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:39:30.745 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:39:30.745 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:39:30.749 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:39:30.749 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:40:58.387 INFO:teuthology.orchestra.run.smithi050.stdout:generic/091 88s 2024-04-02T16:40:58.392 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/091 2024-04-02T16:40:58.393 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:40:58.395 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:40:58.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:40:58.437 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-04-02T16:40:58.815 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:40:58.819 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:40:58.819 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:40:58.820 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:40:58.820 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:40:59.042 INFO:teuthology.orchestra.run.smithi050.stdout:generic/092 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:40:59.046 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/092 2024-04-02T16:40:59.046 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/092 2024-04-02T16:40:59.046 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:40:59.048 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:40:59.062 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:40:59.062 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-04-02T16:40:59.236 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:40:59.240 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:40:59.240 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:40:59.241 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:40:59.241 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:40:59.867 INFO:teuthology.orchestra.run.smithi050.stdout:generic/093 0s 2024-04-02T16:40:59.872 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/093 2024-04-02T16:40:59.872 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:40:59.874 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:40:59.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:40:59.933 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-04-02T16:41:00.314 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:00.317 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:00.318 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:00.319 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:00.319 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:00.565 INFO:teuthology.orchestra.run.smithi050.stdout:generic/094 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-02T16:41:00.569 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/094 2024-04-02T16:41:00.569 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/094 2024-04-02T16:41:00.569 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:00.571 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:00.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:00.625 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-04-02T16:41:01.014 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:01.014 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:01.015 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:01.016 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:01.016 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:10.849 INFO:teuthology.orchestra.run.smithi050.stdout:generic/095 9s 2024-04-02T16:41:10.857 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/095 2024-04-02T16:41:10.857 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:10.862 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:10.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:10.873 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-04-02T16:41:11.114 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:11.117 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:11.118 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:11.119 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:11.119 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:11.393 INFO:teuthology.orchestra.run.smithi050.stdout:generic/096 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-02T16:41:11.397 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/096 2024-04-02T16:41:11.397 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/096 2024-04-02T16:41:11.397 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:11.399 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:11.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:11.416 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-04-02T16:41:11.588 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:11.591 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:11.592 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:11.593 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:11.593 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:12.225 INFO:teuthology.orchestra.run.smithi050.stdout:generic/097 1s 2024-04-02T16:41:12.230 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/097 2024-04-02T16:41:12.230 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:12.232 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:12.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:12.244 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-04-02T16:41:12.430 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:12.434 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:12.434 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:12.435 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:13.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:41:13.752 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:41:14.342 INFO:teuthology.orchestra.run.smithi050.stdout:generic/098 2s 2024-04-02T16:41:14.347 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/098 2024-04-02T16:41:14.347 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:14.349 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:14.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:14.359 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-02T16:41:14.545 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:14.549 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:14.549 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:14.550 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:14.550 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:15.326 INFO:teuthology.orchestra.run.smithi050.stdout:generic/099 1s 2024-04-02T16:41:15.332 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/099 2024-04-02T16:41:15.332 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:15.334 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:15.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:15.390 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-04-02T16:41:15.767 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:15.770 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:15.771 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:15.772 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:15.772 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:26.446 INFO:teuthology.orchestra.run.smithi050.stdout:generic/100 11s 2024-04-02T16:41:26.450 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/100 2024-04-02T16:41:26.450 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:26.453 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:29.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:29.537 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-04-02T16:41:29.906 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:29.909 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:29.910 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:29.911 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:29.911 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:30.118 INFO:teuthology.orchestra.run.smithi050.stdout:generic/101 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:41:30.122 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/101 2024-04-02T16:41:30.122 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/101 2024-04-02T16:41:30.122 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:30.124 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:30.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:30.135 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-04-02T16:41:30.308 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:30.312 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:30.312 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:30.313 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:30.313 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:30.518 INFO:teuthology.orchestra.run.smithi050.stdout:generic/102 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T16:41:30.522 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/102 2024-04-02T16:41:30.522 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/102 2024-04-02T16:41:30.522 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:30.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:30.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:30.535 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-04-02T16:41:30.714 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:30.718 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:30.718 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:30.719 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:30.719 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:30.940 INFO:teuthology.orchestra.run.smithi050.stdout:generic/103 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T16:41:30.943 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/103 2024-04-02T16:41:30.943 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/103 2024-04-02T16:41:30.944 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:30.946 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:30.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:30.993 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-04-02T16:41:31.379 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:31.382 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:31.383 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:31.384 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:31.384 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:31.602 INFO:teuthology.orchestra.run.smithi050.stdout:generic/104 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:41:31.606 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/104 2024-04-02T16:41:31.606 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/104 2024-04-02T16:41:31.606 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:31.608 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:31.621 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:31.621 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-04-02T16:41:31.803 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:31.806 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:31.807 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:31.808 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:31.808 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:32.231 INFO:teuthology.orchestra.run.smithi050.stdout:generic/105 1s 2024-04-02T16:41:32.242 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/105 2024-04-02T16:41:32.242 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:32.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:32.264 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-04-02T16:41:32.567 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:32.571 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:32.572 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:32.572 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:32.573 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:32.792 INFO:teuthology.orchestra.run.smithi050.stdout:generic/106 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:41:32.795 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/106 2024-04-02T16:41:32.795 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/106 2024-04-02T16:41:32.796 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:32.798 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:32.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:32.811 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-04-02T16:41:32.989 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:32.993 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:32.993 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:32.994 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:32.994 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:33.214 INFO:teuthology.orchestra.run.smithi050.stdout:generic/107 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:41:33.218 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/107 2024-04-02T16:41:33.218 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/107 2024-04-02T16:41:33.218 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:33.220 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:33.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:33.233 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-04-02T16:41:33.407 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:33.410 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:33.411 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:33.412 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:33.412 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:33.832 INFO:teuthology.orchestra.run.smithi050.stdout:generic/108 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T16:41:33.836 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/108 2024-04-02T16:41:33.836 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/108 2024-04-02T16:41:33.836 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:33.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:33.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:33.850 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-04-02T16:41:34.024 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:34.028 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:34.028 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:34.029 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:34.029 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:44.007 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:41:44.009 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:41:44.194 INFO:teuthology.orchestra.run.smithi050.stdout:generic/109 10s 2024-04-02T16:41:44.201 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/109 2024-04-02T16:41:44.201 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:41:44.203 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:41:44.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:41:44.211 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-04-02T16:41:44.971 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:41:44.974 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:41:44.975 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:41:44.975 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:41:44.976 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:02.918 INFO:teuthology.orchestra.run.smithi050.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:42:02.925 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/110 2024-04-02T16:42:02.925 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/110 2024-04-02T16:42:02.925 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:02.929 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:03.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:03.022 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-04-02T16:42:03.502 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:03.510 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:03.510 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:03.512 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:03.512 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:03.782 INFO:teuthology.orchestra.run.smithi050.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:42:03.786 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/111 2024-04-02T16:42:03.786 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/111 2024-04-02T16:42:03.786 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:03.788 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:03.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:03.841 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-04-02T16:42:04.267 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:04.270 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:04.271 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:04.272 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:04.272 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:14.066 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:42:14.069 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:42:19.528 INFO:teuthology.orchestra.run.smithi050.stdout:generic/112 15s 2024-04-02T16:42:19.534 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/112 2024-04-02T16:42:19.534 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:19.536 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:19.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:19.600 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-04-02T16:42:19.980 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:19.984 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:19.984 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:19.985 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:19.985 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:34.321 INFO:teuthology.orchestra.run.smithi050.stdout:generic/113 14s 2024-04-02T16:42:34.326 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/113 2024-04-02T16:42:34.326 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:34.328 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:34.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:34.375 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-04-02T16:42:34.878 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:34.884 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:34.888 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:34.891 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:34.892 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:35.122 INFO:teuthology.orchestra.run.smithi050.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-04-02T16:42:35.125 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/114 2024-04-02T16:42:35.126 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/114 2024-04-02T16:42:35.126 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:35.128 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:35.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:35.149 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-04-02T16:42:35.323 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:35.327 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:35.327 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:35.328 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:35.328 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:35.559 INFO:teuthology.orchestra.run.smithi050.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:42:35.563 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/115 2024-04-02T16:42:35.563 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/115 2024-04-02T16:42:35.563 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:35.566 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:35.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:35.629 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-04-02T16:42:35.998 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:36.002 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:36.002 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:36.003 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:36.003 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:36.242 INFO:teuthology.orchestra.run.smithi050.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:42:36.245 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/116 2024-04-02T16:42:36.245 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/116 2024-04-02T16:42:36.245 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:36.247 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:36.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:36.319 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-04-02T16:42:36.698 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:36.702 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:36.702 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:36.703 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:36.703 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:44.153 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:42:44.155 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:42:51.780 INFO:teuthology.orchestra.run.smithi050.stdout:generic/117 15s 2024-04-02T16:42:51.785 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/117 2024-04-02T16:42:51.785 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:51.787 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:51.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:51.796 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-04-02T16:42:51.967 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:51.970 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:51.971 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:51.972 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:51.972 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:52.489 INFO:teuthology.orchestra.run.smithi050.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:42:52.493 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/118 2024-04-02T16:42:52.493 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/118 2024-04-02T16:42:52.493 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:52.496 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:52.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:52.540 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-04-02T16:42:52.979 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:52.985 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:52.986 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:52.987 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:52.987 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:53.214 INFO:teuthology.orchestra.run.smithi050.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:42:53.218 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/119 2024-04-02T16:42:53.218 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/119 2024-04-02T16:42:53.219 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:53.220 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:53.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:53.264 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-04-02T16:42:53.653 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:53.657 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:53.657 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:53.658 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:53.658 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:53.854 INFO:teuthology.orchestra.run.smithi050.stdout:generic/120 [not run] atime not maintained by ceph 2024-04-02T16:42:53.858 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/120 2024-04-02T16:42:53.858 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/120 2024-04-02T16:42:53.858 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:53.860 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:53.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:53.871 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-04-02T16:42:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:54.064 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:54.064 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:54.288 INFO:teuthology.orchestra.run.smithi050.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T16:42:54.292 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/121 2024-04-02T16:42:54.292 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/121 2024-04-02T16:42:54.292 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:54.294 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:54.340 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:54.340 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-04-02T16:42:54.703 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:54.707 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:54.707 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:54.708 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:54.933 INFO:teuthology.orchestra.run.smithi050.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T16:42:54.937 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/122 2024-04-02T16:42:54.937 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/122 2024-04-02T16:42:54.937 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:54.939 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:55.003 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:55.004 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-04-02T16:42:55.373 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:55.376 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:55.377 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:55.378 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:55.378 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:55.865 INFO:teuthology.orchestra.run.smithi050.stdout:generic/123 0s 2024-04-02T16:42:55.870 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/123 2024-04-02T16:42:55.870 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:55.872 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:55.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:55.930 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-04-02T16:42:56.301 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:56.305 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:56.305 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:56.306 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:56.306 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:58.019 INFO:teuthology.orchestra.run.smithi050.stdout:generic/124 2s 2024-04-02T16:42:58.023 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/124 2024-04-02T16:42:58.023 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:42:58.026 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:42:58.479 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:42:58.480 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-04-02T16:42:58.953 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:42:58.958 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:42:58.959 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:42:58.960 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:42:58.961 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:43:14.366 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:43:14.369 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:43:44.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:43:44.454 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:44:00.377 INFO:teuthology.orchestra.run.smithi050.stdout:generic/125 61s 2024-04-02T16:44:00.383 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/125 2024-04-02T16:44:00.383 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:44:00.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:44:00.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:44:00.427 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-04-02T16:44:00.805 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:44:00.808 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:44:00.809 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:44:00.809 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:44:00.810 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:44:01.199 INFO:teuthology.orchestra.run.smithi050.stdout:generic/126 1s 2024-04-02T16:44:01.204 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/126 2024-04-02T16:44:01.204 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:44:01.206 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:44:01.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:44:01.271 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-04-02T16:44:01.680 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:44:01.683 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:44:01.684 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:44:01.685 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:44:01.685 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:44:14.604 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:44:14.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:44:44.707 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:44:44.710 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:45:14.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:45:14.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:45:45.017 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:45:45.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:46:15.176 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:46:15.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:46:45.245 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:46:45.249 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:47:15.589 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:47:15.592 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:47:45.945 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:47:45.948 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:48:16.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:48:16.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:48:46.275 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:48:46.278 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:49:16.331 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:49:16.334 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:49:46.424 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:49:46.427 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:50:16.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:50:16.489 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:50:46.579 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:50:46.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:51:16.667 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:51:16.703 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:51:46.765 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:51:46.768 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:52:16.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:52:16.906 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:52:40.533 INFO:teuthology.orchestra.run.smithi050.stdout:generic/127 519s 2024-04-02T16:52:40.537 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/127 2024-04-02T16:52:40.538 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:52:40.540 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:40.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:52:40.587 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-04-02T16:52:40.952 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:52:40.955 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:52:40.956 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:52:40.957 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:52:40.957 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:42.004 INFO:teuthology.orchestra.run.smithi050.stdout:generic/128 0s 2024-04-02T16:52:42.010 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/128 2024-04-02T16:52:42.010 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:52:42.013 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:42.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:52:42.022 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-04-02T16:52:42.204 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:52:42.208 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:52:42.208 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:52:42.209 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:52:42.209 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:42.433 INFO:teuthology.orchestra.run.smithi050.stdout:generic/129 [not run] Sparse files are not supported or do not work as expected 2024-04-02T16:52:42.437 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/129 2024-04-02T16:52:42.437 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/129 2024-04-02T16:52:42.437 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:52:42.439 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:42.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:52:42.460 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-04-02T16:52:42.637 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:52:42.641 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:52:42.641 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:52:42.642 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:52:42.642 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:42.858 INFO:teuthology.orchestra.run.smithi050.stdout:generic/130 [not run] Sparse files are not supported or do not work as expected 2024-04-02T16:52:42.861 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/130 2024-04-02T16:52:42.861 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/130 2024-04-02T16:52:42.861 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:52:42.864 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:42.884 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:52:42.885 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-04-02T16:52:43.059 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:52:43.062 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:52:43.063 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:52:43.064 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:52:43.064 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:44.406 INFO:teuthology.orchestra.run.smithi050.stdout:generic/131 1s 2024-04-02T16:52:44.412 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/131 2024-04-02T16:52:44.412 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:52:44.414 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:44.484 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:52:44.484 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-04-02T16:52:44.860 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:52:44.864 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:52:44.864 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:52:44.865 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:52:44.865 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:47.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:52:47.037 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:52:57.142 INFO:teuthology.orchestra.run.smithi050.stdout:generic/132 13s 2024-04-02T16:52:57.147 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/132 2024-04-02T16:52:57.147 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:52:57.149 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:52:57.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:52:57.157 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-04-02T16:52:57.337 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:52:57.340 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:52:57.341 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:52:57.342 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:52:57.342 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:53:17.197 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:53:17.199 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:53:47.358 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:53:47.361 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:54:17.603 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:54:17.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:54:47.722 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:54:47.726 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:55:17.934 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:55:17.937 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:55:48.304 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:55:48.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:56:18.382 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:56:18.385 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:56:28.297 INFO:teuthology.orchestra.run.smithi050.stdout:generic/133 211s 2024-04-02T16:56:28.301 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/133 2024-04-02T16:56:28.301 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:28.341 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:28.341 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-04-02T16:56:28.733 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:28.737 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:28.737 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:28.738 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:28.738 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:29.058 INFO:teuthology.orchestra.run.smithi050.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:29.062 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/134 2024-04-02T16:56:29.062 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/134 2024-04-02T16:56:29.062 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:29.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:29.120 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-04-02T16:56:29.504 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:29.508 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:29.509 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:29.510 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:29.510 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:30.287 INFO:teuthology.orchestra.run.smithi050.stdout:generic/135 1s 2024-04-02T16:56:30.292 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/135 2024-04-02T16:56:30.292 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:30.294 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:30.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:30.303 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-04-02T16:56:30.479 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:30.483 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:30.483 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:30.484 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:30.484 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:30.713 INFO:teuthology.orchestra.run.smithi050.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T16:56:30.717 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/136 2024-04-02T16:56:30.717 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/136 2024-04-02T16:56:30.717 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:30.719 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:30.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:30.767 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-04-02T16:56:31.152 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:31.156 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:31.156 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:31.157 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:31.157 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:31.392 INFO:teuthology.orchestra.run.smithi050.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:31.396 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/137 2024-04-02T16:56:31.396 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/137 2024-04-02T16:56:31.396 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:31.398 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:31.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:31.449 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-04-02T16:56:31.808 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:31.812 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:31.813 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:31.813 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:31.814 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:32.036 INFO:teuthology.orchestra.run.smithi050.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:32.040 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/138 2024-04-02T16:56:32.040 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/138 2024-04-02T16:56:32.040 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:32.091 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:32.091 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-04-02T16:56:32.466 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:32.470 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:32.470 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:32.471 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:32.471 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:32.696 INFO:teuthology.orchestra.run.smithi050.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:32.700 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/139 2024-04-02T16:56:32.700 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/139 2024-04-02T16:56:32.700 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:32.702 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:32.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:32.753 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-04-02T16:56:33.133 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:33.137 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:33.137 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:33.138 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:33.138 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:33.363 INFO:teuthology.orchestra.run.smithi050.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:33.367 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/140 2024-04-02T16:56:33.367 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/140 2024-04-02T16:56:33.367 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:33.369 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:33.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:33.419 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-04-02T16:56:33.805 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:33.809 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:33.810 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:33.810 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:34.134 INFO:teuthology.orchestra.run.smithi050.stdout:generic/141 1s 2024-04-02T16:56:34.139 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/141 2024-04-02T16:56:34.139 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:34.141 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:34.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:34.149 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-04-02T16:56:34.320 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:34.324 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:34.324 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:34.325 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:34.325 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:34.559 INFO:teuthology.orchestra.run.smithi050.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:34.563 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/142 2024-04-02T16:56:34.563 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/142 2024-04-02T16:56:34.563 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:34.566 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:34.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:34.619 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-04-02T16:56:35.002 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:35.005 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:35.006 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:35.006 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:35.007 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:35.233 INFO:teuthology.orchestra.run.smithi050.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:35.237 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/143 2024-04-02T16:56:35.237 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/143 2024-04-02T16:56:35.237 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:35.239 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:35.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:35.285 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-04-02T16:56:35.637 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:35.641 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:35.642 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:35.643 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:35.643 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:35.868 INFO:teuthology.orchestra.run.smithi050.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:35.872 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/144 2024-04-02T16:56:35.872 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/144 2024-04-02T16:56:35.872 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:35.874 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:35.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:35.930 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-04-02T16:56:36.311 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:36.314 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:36.315 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:36.316 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:36.316 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:36.543 INFO:teuthology.orchestra.run.smithi050.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:36.547 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/145 2024-04-02T16:56:36.547 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/145 2024-04-02T16:56:36.547 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:36.549 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:36.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:36.601 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-04-02T16:56:36.962 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:36.966 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:36.966 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:36.967 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:36.967 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:37.197 INFO:teuthology.orchestra.run.smithi050.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:37.200 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/146 2024-04-02T16:56:37.200 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/146 2024-04-02T16:56:37.200 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:37.203 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:37.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:37.252 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-04-02T16:56:37.632 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:37.636 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:37.636 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:37.637 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:37.637 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:37.873 INFO:teuthology.orchestra.run.smithi050.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:37.877 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/147 2024-04-02T16:56:37.877 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/147 2024-04-02T16:56:37.877 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:37.879 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:37.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:37.929 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-04-02T16:56:38.310 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:38.314 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:38.315 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:38.315 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:38.315 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:38.544 INFO:teuthology.orchestra.run.smithi050.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:38.547 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/148 2024-04-02T16:56:38.547 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/148 2024-04-02T16:56:38.548 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:38.550 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:38.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:38.599 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-04-02T16:56:38.992 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:38.996 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:38.996 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:38.997 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:38.997 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:39.227 INFO:teuthology.orchestra.run.smithi050.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:39.230 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/149 2024-04-02T16:56:39.231 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/149 2024-04-02T16:56:39.231 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:39.233 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:39.283 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:39.283 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-04-02T16:56:39.659 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:39.663 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:39.663 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:39.664 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:39.664 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:39.887 INFO:teuthology.orchestra.run.smithi050.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:39.891 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/150 2024-04-02T16:56:39.891 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/150 2024-04-02T16:56:39.891 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:39.893 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:39.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:39.946 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-04-02T16:56:40.317 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:40.320 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:40.320 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:40.321 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:40.322 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:40.553 INFO:teuthology.orchestra.run.smithi050.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:40.556 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/151 2024-04-02T16:56:40.556 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/151 2024-04-02T16:56:40.557 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:40.559 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:40.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:40.613 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-04-02T16:56:40.969 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:40.973 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:40.973 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:40.974 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:40.974 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:41.213 INFO:teuthology.orchestra.run.smithi050.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:41.216 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/152 2024-04-02T16:56:41.216 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/152 2024-04-02T16:56:41.217 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:41.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:41.285 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-04-02T16:56:41.651 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:41.655 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:41.655 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:41.656 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:41.656 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:41.880 INFO:teuthology.orchestra.run.smithi050.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:41.884 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/153 2024-04-02T16:56:41.884 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/153 2024-04-02T16:56:41.884 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:41.886 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:41.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:41.941 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-04-02T16:56:42.321 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:42.324 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:42.325 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:42.326 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:42.326 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:42.556 INFO:teuthology.orchestra.run.smithi050.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:42.559 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/154 2024-04-02T16:56:42.560 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/154 2024-04-02T16:56:42.560 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:42.562 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:42.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:42.613 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-04-02T16:56:43.003 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:43.006 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:43.006 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:43.007 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:43.007 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:43.244 INFO:teuthology.orchestra.run.smithi050.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:43.248 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/155 2024-04-02T16:56:43.248 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/155 2024-04-02T16:56:43.248 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:43.250 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:43.318 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:43.319 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-04-02T16:56:43.695 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:43.699 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:43.699 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:43.700 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:43.700 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:43.926 INFO:teuthology.orchestra.run.smithi050.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:43.930 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/156 2024-04-02T16:56:43.930 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/156 2024-04-02T16:56:43.930 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:43.932 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:43.980 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:43.981 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-04-02T16:56:44.363 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:44.367 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:44.367 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:44.368 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:44.368 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:44.606 INFO:teuthology.orchestra.run.smithi050.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:56:44.610 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/157 2024-04-02T16:56:44.610 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/157 2024-04-02T16:56:44.610 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:44.612 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:44.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:44.673 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-04-02T16:56:45.049 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:45.052 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:45.053 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:45.054 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:45.054 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:45.280 INFO:teuthology.orchestra.run.smithi050.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T16:56:45.284 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/158 2024-04-02T16:56:45.284 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/158 2024-04-02T16:56:45.284 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:45.286 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:45.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:45.347 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-04-02T16:56:45.707 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:45.711 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:45.711 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:45.712 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:45.712 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:45.946 INFO:teuthology.orchestra.run.smithi050.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-04-02T16:56:45.949 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/159 2024-04-02T16:56:45.950 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/159 2024-04-02T16:56:45.950 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:45.952 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:45.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:45.990 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-04-02T16:56:46.160 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:46.163 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:46.164 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:46.165 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:46.165 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:46.381 INFO:teuthology.orchestra.run.smithi050.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-04-02T16:56:46.386 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/160 2024-04-02T16:56:46.386 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/160 2024-04-02T16:56:46.386 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:46.388 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:46.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:46.404 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-04-02T16:56:46.580 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:46.584 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:46.584 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:46.585 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:46.585 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:46.904 INFO:teuthology.orchestra.run.smithi050.stdout:generic/161 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/161 2024-04-02T16:56:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/161 2024-04-02T16:56:46.910 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:46.913 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:47.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:47.202 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-04-02T16:56:47.381 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:47.385 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:47.385 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:47.386 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:47.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:47.680 INFO:teuthology.orchestra.run.smithi050.stdout:generic/162 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-04-02T16:56:47.725 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/162 2024-04-02T16:56:47.725 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/162 2024-04-02T16:56:47.725 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:47.728 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:48.044 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:48.044 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-04-02T16:56:48.218 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:48.222 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:48.222 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:48.223 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:48.223 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:48.452 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:56:48.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:56:48.534 INFO:teuthology.orchestra.run.smithi050.stdout:generic/163 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-04-02T16:56:48.603 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/163 2024-04-02T16:56:48.603 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/163 2024-04-02T16:56:48.604 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:48.607 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:48.898 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:48.899 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-04-02T16:56:49.118 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:49.122 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:49.122 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:49.123 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:49.123 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:49.467 INFO:teuthology.orchestra.run.smithi050.stdout:generic/164 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:49.474 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/164 2024-04-02T16:56:49.474 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/164 2024-04-02T16:56:49.474 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:49.475 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:49.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:49.747 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-04-02T16:56:49.930 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:49.933 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:49.934 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:49.935 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:49.935 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:50.268 INFO:teuthology.orchestra.run.smithi050.stdout:generic/165 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:50.278 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/165 2024-04-02T16:56:50.278 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/165 2024-04-02T16:56:50.278 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:50.284 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:50.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:50.522 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-04-02T16:56:50.717 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:50.720 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:50.721 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:50.722 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:50.722 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:51.061 INFO:teuthology.orchestra.run.smithi050.stdout:generic/166 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:51.068 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/166 2024-04-02T16:56:51.068 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/166 2024-04-02T16:56:51.068 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:51.070 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:51.379 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:51.379 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-04-02T16:56:51.557 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:51.560 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:51.560 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:51.561 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:51.561 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:51.891 INFO:teuthology.orchestra.run.smithi050.stdout:generic/167 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:51.896 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/167 2024-04-02T16:56:51.896 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/167 2024-04-02T16:56:51.896 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:51.900 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:52.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:52.168 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-04-02T16:56:52.353 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:52.356 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:52.357 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:52.358 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:52.358 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:52.721 INFO:teuthology.orchestra.run.smithi050.stdout:generic/168 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:52.732 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/168 2024-04-02T16:56:52.732 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/168 2024-04-02T16:56:52.732 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:52.735 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:53.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:53.051 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-04-02T16:56:53.223 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:53.226 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:53.227 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:53.227 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:53.228 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:53.641 INFO:teuthology.orchestra.run.smithi050.stdout:generic/169 0s 2024-04-02T16:56:53.646 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/169 2024-04-02T16:56:53.647 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:53.649 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:53.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:53.657 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-04-02T16:56:53.835 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:53.838 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:53.839 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:53.840 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:53.840 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:54.175 INFO:teuthology.orchestra.run.smithi050.stdout:generic/170 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:54.184 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/170 2024-04-02T16:56:54.184 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/170 2024-04-02T16:56:54.184 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:54.189 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:54.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:54.475 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-04-02T16:56:54.655 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:54.659 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:54.659 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:54.660 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:54.661 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:55.015 INFO:teuthology.orchestra.run.smithi050.stdout:generic/171 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:55.020 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/171 2024-04-02T16:56:55.021 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/171 2024-04-02T16:56:55.021 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:55.028 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:55.299 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:55.300 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-04-02T16:56:55.498 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:55.501 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:55.502 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:55.503 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:55.503 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:55.857 INFO:teuthology.orchestra.run.smithi050.stdout:generic/172 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:55.864 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/172 2024-04-02T16:56:55.864 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/172 2024-04-02T16:56:55.864 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:55.866 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:56.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:56.156 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-04-02T16:56:56.334 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:56.338 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:56.338 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:56.339 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:56.339 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:56.682 INFO:teuthology.orchestra.run.smithi050.stdout:generic/173 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:56.691 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/173 2024-04-02T16:56:56.692 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/173 2024-04-02T16:56:56.692 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:56.696 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:56.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:56.893 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-04-02T16:56:57.121 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:57.124 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:57.126 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:57.468 INFO:teuthology.orchestra.run.smithi050.stdout:generic/174 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:57.475 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/174 2024-04-02T16:56:57.475 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/174 2024-04-02T16:56:57.475 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:57.478 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:57.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:57.810 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-04-02T16:56:57.996 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:58.000 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:58.000 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:58.001 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:58.002 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:58.350 INFO:teuthology.orchestra.run.smithi050.stdout:generic/175 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:58.353 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/175 2024-04-02T16:56:58.353 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/175 2024-04-02T16:56:58.353 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:58.356 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:58.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:58.679 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-04-02T16:56:58.861 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:58.864 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:58.865 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:58.866 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:58.866 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:59.219 INFO:teuthology.orchestra.run.smithi050.stdout:generic/176 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:56:59.226 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/176 2024-04-02T16:56:59.226 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/176 2024-04-02T16:56:59.226 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:56:59.228 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:56:59.528 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:56:59.529 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-04-02T16:56:59.715 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:56:59.718 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:56:59.719 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:56:59.719 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:56:59.720 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:00.065 INFO:teuthology.orchestra.run.smithi050.stdout:generic/177 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-02T16:57:00.068 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/177 2024-04-02T16:57:00.068 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/177 2024-04-02T16:57:00.069 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:00.071 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:00.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:00.111 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-04-02T16:57:00.493 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:00.496 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:00.497 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:00.498 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:00.498 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:00.732 INFO:teuthology.orchestra.run.smithi050.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:57:00.735 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/178 2024-04-02T16:57:00.735 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/178 2024-04-02T16:57:00.736 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:00.738 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:00.806 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:00.807 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-04-02T16:57:01.166 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:01.169 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:01.169 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:01.170 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:01.170 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:01.401 INFO:teuthology.orchestra.run.smithi050.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:57:01.405 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/179 2024-04-02T16:57:01.405 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/179 2024-04-02T16:57:01.405 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:01.407 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:01.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:01.449 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-04-02T16:57:01.819 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:01.823 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:01.823 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:01.824 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:01.824 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:02.050 INFO:teuthology.orchestra.run.smithi050.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:57:02.054 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/180 2024-04-02T16:57:02.054 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/180 2024-04-02T16:57:02.054 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:02.056 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:02.106 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:02.106 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-04-02T16:57:02.469 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:02.473 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:02.473 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:02.474 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:02.474 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:02.699 INFO:teuthology.orchestra.run.smithi050.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T16:57:02.703 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/181 2024-04-02T16:57:02.703 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/181 2024-04-02T16:57:02.703 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:02.705 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:02.766 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:02.766 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-04-02T16:57:03.158 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:03.161 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:03.162 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:03.163 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:03.163 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:03.394 INFO:teuthology.orchestra.run.smithi050.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T16:57:03.397 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/182 2024-04-02T16:57:03.398 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/182 2024-04-02T16:57:03.398 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:03.400 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:03.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:03.454 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-04-02T16:57:03.838 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:03.841 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:03.842 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:03.843 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:03.843 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:04.170 INFO:teuthology.orchestra.run.smithi050.stdout:generic/183 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:04.175 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/183 2024-04-02T16:57:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/183 2024-04-02T16:57:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:04.179 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:04.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:04.477 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-04-02T16:57:04.659 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:04.662 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:04.663 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:04.664 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:04.664 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:04.914 INFO:teuthology.orchestra.run.smithi050.stdout:generic/184 0s 2024-04-02T16:57:04.919 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/184 2024-04-02T16:57:04.919 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:04.921 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:04.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:04.938 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-04-02T16:57:05.118 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:05.122 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:05.122 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:05.124 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:05.124 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:05.496 INFO:teuthology.orchestra.run.smithi050.stdout:generic/185 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/185 2024-04-02T16:57:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/185 2024-04-02T16:57:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:05.506 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:05.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:05.742 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-04-02T16:57:05.925 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:05.929 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:05.929 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:05.930 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:05.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:06.273 INFO:teuthology.orchestra.run.smithi050.stdout:generic/186 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:06.282 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/186 2024-04-02T16:57:06.282 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/186 2024-04-02T16:57:06.283 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:06.288 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:06.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:06.561 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-04-02T16:57:06.738 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:06.741 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:06.742 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:06.742 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:06.743 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:07.075 INFO:teuthology.orchestra.run.smithi050.stdout:generic/187 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:07.086 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/187 2024-04-02T16:57:07.086 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/187 2024-04-02T16:57:07.086 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:07.087 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:07.331 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:07.332 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-04-02T16:57:07.533 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:07.536 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:07.537 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:07.538 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:07.538 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:07.883 INFO:teuthology.orchestra.run.smithi050.stdout:generic/188 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:07.888 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/188 2024-04-02T16:57:07.888 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/188 2024-04-02T16:57:07.888 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:08.200 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:08.200 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-04-02T16:57:08.377 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:08.380 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:08.381 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:08.381 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:08.721 INFO:teuthology.orchestra.run.smithi050.stdout:generic/189 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:08.726 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/189 2024-04-02T16:57:08.726 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/189 2024-04-02T16:57:08.726 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:08.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:08.972 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-04-02T16:57:09.172 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:09.175 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:09.175 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:09.176 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:09.176 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:09.523 INFO:teuthology.orchestra.run.smithi050.stdout:generic/190 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:09.529 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/190 2024-04-02T16:57:09.530 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/190 2024-04-02T16:57:09.530 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:09.532 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:09.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:09.798 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-04-02T16:57:09.983 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:09.987 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:09.987 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:09.988 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:09.988 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:10.328 INFO:teuthology.orchestra.run.smithi050.stdout:generic/191 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:10.335 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/191 2024-04-02T16:57:10.335 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/191 2024-04-02T16:57:10.335 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:10.343 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:10.594 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:10.594 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-04-02T16:57:10.788 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:10.791 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:10.792 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:10.792 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:10.793 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:11.009 INFO:teuthology.orchestra.run.smithi050.stdout:generic/192 [not run] atime not maintained by ceph 2024-04-02T16:57:11.013 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/192 2024-04-02T16:57:11.013 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/192 2024-04-02T16:57:11.013 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:11.015 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:11.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:11.029 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-04-02T16:57:11.202 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:11.206 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:11.206 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:11.207 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:11.207 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:12.103 INFO:teuthology.orchestra.run.smithi050.stdout:generic/193 1s 2024-04-02T16:57:12.108 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/193 2024-04-02T16:57:12.108 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:12.110 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:12.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:12.163 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-04-02T16:57:12.541 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:12.545 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:12.545 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:12.546 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:12.546 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:12.880 INFO:teuthology.orchestra.run.smithi050.stdout:generic/194 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:12.886 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/194 2024-04-02T16:57:12.887 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/194 2024-04-02T16:57:12.887 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:12.889 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:13.177 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:13.177 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-04-02T16:57:13.365 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:13.368 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:13.369 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:13.369 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:13.370 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:13.732 INFO:teuthology.orchestra.run.smithi050.stdout:generic/195 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:13.739 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/195 2024-04-02T16:57:13.739 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/195 2024-04-02T16:57:13.739 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:13.741 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:13.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:13.997 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-04-02T16:57:14.181 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:14.185 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:14.185 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:14.186 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:14.186 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:14.541 INFO:teuthology.orchestra.run.smithi050.stdout:generic/196 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:14.546 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/196 2024-04-02T16:57:14.546 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/196 2024-04-02T16:57:14.546 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:14.551 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:14.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:14.842 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-04-02T16:57:15.020 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:15.023 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:15.024 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:15.025 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:15.025 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:15.369 INFO:teuthology.orchestra.run.smithi050.stdout:generic/197 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:15.374 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/197 2024-04-02T16:57:15.374 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/197 2024-04-02T16:57:15.374 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:15.378 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:15.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:15.654 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-04-02T16:57:15.842 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:15.846 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:15.846 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:15.847 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:15.847 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:16.460 INFO:teuthology.orchestra.run.smithi050.stdout:generic/198 1s 2024-04-02T16:57:16.465 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/198 2024-04-02T16:57:16.465 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:16.468 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:16.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:16.518 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-04-02T16:57:16.908 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:16.911 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:16.912 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:16.913 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:16.913 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:17.284 INFO:teuthology.orchestra.run.smithi050.stdout:generic/199 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:17.290 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/199 2024-04-02T16:57:17.290 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/199 2024-04-02T16:57:17.290 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:17.293 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:17.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:17.535 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-04-02T16:57:17.742 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:17.745 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:17.746 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:17.746 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:17.746 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:18.080 INFO:teuthology.orchestra.run.smithi050.stdout:generic/200 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:18.086 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/200 2024-04-02T16:57:18.086 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/200 2024-04-02T16:57:18.086 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:18.088 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:18.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:18.300 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-04-02T16:57:18.521 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:18.525 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:18.525 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:18.526 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:18.526 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:18.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:57:18.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:57:18.869 INFO:teuthology.orchestra.run.smithi050.stdout:generic/201 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:18.875 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/201 2024-04-02T16:57:18.875 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/201 2024-04-02T16:57:18.875 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:18.878 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:19.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:19.143 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-04-02T16:57:19.330 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:19.333 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:19.334 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:19.335 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:19.676 INFO:teuthology.orchestra.run.smithi050.stdout:generic/202 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:19.683 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/202 2024-04-02T16:57:19.683 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/202 2024-04-02T16:57:19.683 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:19.685 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:19.981 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:19.981 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-04-02T16:57:20.159 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:20.163 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:20.163 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:20.164 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:20.164 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:20.517 INFO:teuthology.orchestra.run.smithi050.stdout:generic/203 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:20.524 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/203 2024-04-02T16:57:20.524 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/203 2024-04-02T16:57:20.524 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:20.526 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:20.754 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:20.754 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-04-02T16:57:20.960 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:20.964 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:20.964 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:20.965 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:20.965 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:23.364 INFO:teuthology.orchestra.run.smithi050.stdout:generic/204 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T16:57:23.369 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/204 2024-04-02T16:57:23.369 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/204 2024-04-02T16:57:23.369 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:23.371 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:23.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:23.392 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-04-02T16:57:23.574 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:23.577 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:23.578 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:23.579 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:23.579 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:generic/205 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:23.926 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/205 2024-04-02T16:57:23.926 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/205 2024-04-02T16:57:23.927 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:23.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:24.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:24.143 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-04-02T16:57:24.352 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:24.356 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:24.356 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:24.357 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:24.357 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:24.704 INFO:teuthology.orchestra.run.smithi050.stdout:generic/206 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T16:57:24.710 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/206 2024-04-02T16:57:24.711 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/206 2024-04-02T16:57:24.711 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:24.713 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:25.002 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:25.002 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-04-02T16:57:25.181 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:25.185 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:25.185 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:25.186 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:25.186 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:31.130 INFO:teuthology.orchestra.run.smithi050.stdout:generic/207 6s 2024-04-02T16:57:31.137 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/207 2024-04-02T16:57:31.137 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T16:57:31.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:31.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T16:57:31.187 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-04-02T16:57:31.558 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T16:57:31.561 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T16:57:31.561 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T16:57:31.562 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T16:57:31.562 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T16:57:48.633 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:57:48.637 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:58:18.746 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:58:18.748 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:58:48.908 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:58:48.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:58:48.955 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T16:58:48.956+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 247244) UID: 0 2024-04-02T16:58:48.955 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T16:58:48.956+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 247244) UID: 0 2024-04-02T16:58:48.956 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T16:58:48.956+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 247244) UID: 0 2024-04-02T16:58:48.957 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T16:58:48.957+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 247244) UID: 0 2024-04-02T17:00:51.878 INFO:teuthology.orchestra.run.smithi050.stdout:generic/208 200s 2024-04-02T17:00:51.883 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/208 2024-04-02T17:00:51.883 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:00:51.885 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:00:51.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:00:51.935 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-04-02T17:00:52.363 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:00:52.366 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:00:52.367 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:00:52.368 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:00:52.368 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:27.894 INFO:teuthology.orchestra.run.smithi050.stdout:generic/209 35s 2024-04-02T17:01:27.898 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/209 2024-04-02T17:01:27.898 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:27.901 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:27.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:27.945 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-04-02T17:01:28.344 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:28.349 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:28.349 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:28.350 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:28.350 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:28.651 INFO:teuthology.orchestra.run.smithi050.stdout:generic/210 0s 2024-04-02T17:01:28.656 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/210 2024-04-02T17:01:28.657 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:28.659 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:28.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:28.704 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-04-02T17:01:29.111 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:29.114 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:29.115 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:29.115 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:29.116 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:29.485 INFO:teuthology.orchestra.run.smithi050.stdout:generic/211 0s 2024-04-02T17:01:29.490 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/211 2024-04-02T17:01:29.491 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:29.493 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:29.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:29.509 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-04-02T17:01:29.691 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:29.695 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:29.695 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:29.697 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:29.697 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:29.986 INFO:teuthology.orchestra.run.smithi050.stdout:generic/212 0s 2024-04-02T17:01:29.991 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/212 2024-04-02T17:01:29.992 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:29.993 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:30.010 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:30.010 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-04-02T17:01:30.188 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:30.191 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:30.192 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:30.193 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:30.193 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:30.415 INFO:teuthology.orchestra.run.smithi050.stdout:generic/213 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:01:30.419 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/213 2024-04-02T17:01:30.419 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/213 2024-04-02T17:01:30.419 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:30.421 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:30.434 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:30.434 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-04-02T17:01:30.611 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:30.614 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:30.615 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:30.616 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:30.616 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:30.834 INFO:teuthology.orchestra.run.smithi050.stdout:generic/214 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:01:30.838 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/214 2024-04-02T17:01:30.838 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/214 2024-04-02T17:01:30.838 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:30.853 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:30.853 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-04-02T17:01:31.041 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:31.046 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:31.046 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:33.628 INFO:teuthology.orchestra.run.smithi050.stdout:generic/215 2s 2024-04-02T17:01:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/215 2024-04-02T17:01:33.633 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:33.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:33.668 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-04-02T17:01:33.847 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:33.851 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:33.851 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:33.852 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:33.852 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:34.185 INFO:teuthology.orchestra.run.smithi050.stdout:generic/216 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:34.200 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/216 2024-04-02T17:01:34.200 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/216 2024-04-02T17:01:34.201 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:34.203 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:34.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:34.497 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-04-02T17:01:34.694 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:34.698 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:34.698 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:34.699 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:34.699 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:35.054 INFO:teuthology.orchestra.run.smithi050.stdout:generic/217 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:35.059 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/217 2024-04-02T17:01:35.059 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/217 2024-04-02T17:01:35.059 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:35.062 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:35.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:35.302 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-04-02T17:01:35.537 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:35.541 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:35.541 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:35.542 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:35.542 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:35.888 INFO:teuthology.orchestra.run.smithi050.stdout:generic/218 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:35.894 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/218 2024-04-02T17:01:35.894 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/218 2024-04-02T17:01:35.894 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:35.898 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:36.175 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:36.175 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-04-02T17:01:36.383 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:36.387 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:36.387 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:36.388 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:36.389 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:36.593 INFO:teuthology.orchestra.run.smithi050.stdout:generic/219 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/219 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/219 2024-04-02T17:01:36.597 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:36.599 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:36.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:36.612 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-04-02T17:01:36.797 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:36.800 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:36.801 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:36.802 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:36.802 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:37.158 INFO:teuthology.orchestra.run.smithi050.stdout:generic/220 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/220 2024-04-02T17:01:37.163 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/220 2024-04-02T17:01:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:37.167 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:37.455 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:37.455 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-04-02T17:01:37.659 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:37.663 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:37.663 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:37.664 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:37.664 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:38.892 INFO:teuthology.orchestra.run.smithi050.stdout:generic/221 1s 2024-04-02T17:01:38.897 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/221 2024-04-02T17:01:38.897 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:38.900 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:38.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:38.921 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-04-02T17:01:39.103 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:39.107 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:39.107 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:39.108 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:39.109 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:39.454 INFO:teuthology.orchestra.run.smithi050.stdout:generic/222 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:39.460 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/222 2024-04-02T17:01:39.460 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/222 2024-04-02T17:01:39.460 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:39.463 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:39.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:39.774 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-04-02T17:01:39.968 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:39.972 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:39.972 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:39.973 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:39.973 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:40.184 INFO:teuthology.orchestra.run.smithi050.stdout:generic/223 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:01:40.188 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/223 2024-04-02T17:01:40.188 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/223 2024-04-02T17:01:40.188 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:40.190 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:40.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:40.207 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-04-02T17:01:40.383 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:40.386 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:40.387 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:40.388 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:40.388 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:40.583 INFO:teuthology.orchestra.run.smithi050.stdout:generic/224 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:01:40.587 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/224 2024-04-02T17:01:40.587 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/224 2024-04-02T17:01:40.588 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:40.589 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:40.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:40.602 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-04-02T17:01:40.779 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:40.783 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:40.783 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:40.784 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:40.784 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:41.042 INFO:teuthology.orchestra.run.smithi050.stdout:generic/225 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-02T17:01:41.046 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/225 2024-04-02T17:01:41.046 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/225 2024-04-02T17:01:41.046 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:41.048 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:41.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:41.096 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-04-02T17:01:41.518 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:41.522 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:41.522 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:41.523 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:41.523 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:41.761 INFO:teuthology.orchestra.run.smithi050.stdout:generic/226 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:01:41.765 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/226 2024-04-02T17:01:41.765 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/226 2024-04-02T17:01:41.765 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:41.767 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:41.783 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:41.783 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-04-02T17:01:41.955 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:41.959 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:41.959 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:41.960 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:41.960 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:42.308 INFO:teuthology.orchestra.run.smithi050.stdout:generic/227 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:42.313 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/227 2024-04-02T17:01:42.313 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/227 2024-04-02T17:01:42.313 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:42.316 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:42.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:42.567 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-04-02T17:01:42.795 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:42.799 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:42.799 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:42.800 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:42.800 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:43.045 INFO:teuthology.orchestra.run.smithi050.stdout:generic/228 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:01:43.049 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/228 2024-04-02T17:01:43.049 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/228 2024-04-02T17:01:43.049 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:43.051 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:43.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:43.064 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-04-02T17:01:43.237 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:43.240 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:43.241 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:43.241 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:43.241 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:43.586 INFO:teuthology.orchestra.run.smithi050.stdout:generic/229 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:43.593 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/229 2024-04-02T17:01:43.594 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/229 2024-04-02T17:01:43.594 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:43.597 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:43.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:43.876 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-04-02T17:01:44.080 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:44.083 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:44.084 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:44.085 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:44.085 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:44.330 INFO:teuthology.orchestra.run.smithi050.stdout:generic/230 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:01:44.334 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/230 2024-04-02T17:01:44.334 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/230 2024-04-02T17:01:44.335 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:44.337 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:44.352 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:44.352 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-04-02T17:01:44.546 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:44.550 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:44.551 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:44.552 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:44.552 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:44.775 INFO:teuthology.orchestra.run.smithi050.stdout:generic/231 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:01:44.779 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/231 2024-04-02T17:01:44.780 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/231 2024-04-02T17:01:44.780 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:44.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:44.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:44.810 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-04-02T17:01:45.026 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:45.029 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:45.030 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:45.031 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:45.031 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:45.231 INFO:teuthology.orchestra.run.smithi050.stdout:generic/232 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:01:45.235 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/232 2024-04-02T17:01:45.235 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/232 2024-04-02T17:01:45.235 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:45.237 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:45.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:45.254 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-04-02T17:01:45.427 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:45.430 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:45.431 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:45.432 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:45.432 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:45.629 INFO:teuthology.orchestra.run.smithi050.stdout:generic/233 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:01:45.632 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/233 2024-04-02T17:01:45.632 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/233 2024-04-02T17:01:45.632 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:45.635 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:45.648 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:45.648 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-04-02T17:01:45.831 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:45.834 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:45.835 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:45.836 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:45.836 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:46.037 INFO:teuthology.orchestra.run.smithi050.stdout:generic/234 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:01:46.040 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/234 2024-04-02T17:01:46.040 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/234 2024-04-02T17:01:46.040 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:46.043 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:46.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:46.056 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-04-02T17:01:46.244 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:46.249 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:46.249 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:46.249 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:46.249 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:46.450 INFO:teuthology.orchestra.run.smithi050.stdout:generic/235 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:01:46.453 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/235 2024-04-02T17:01:46.454 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/235 2024-04-02T17:01:46.454 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:46.456 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:46.479 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:46.479 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-04-02T17:01:46.657 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:46.661 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:46.662 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:46.663 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:46.663 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:47.934 INFO:teuthology.orchestra.run.smithi050.stdout:generic/236 1s 2024-04-02T17:01:47.940 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/236 2024-04-02T17:01:47.940 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:47.943 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:47.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:47.988 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-04-02T17:01:48.375 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:48.378 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:48.378 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:48.379 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:48.380 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:48.687 INFO:teuthology.orchestra.run.smithi050.stdout:generic/237 0s 2024-04-02T17:01:48.693 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/237 2024-04-02T17:01:48.693 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:48.695 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:48.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:48.747 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-04-02T17:01:49.116 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:49.120 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:49.120 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:49.121 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:49.121 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:49.454 INFO:teuthology.orchestra.run.smithi050.stdout:generic/238 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:01:49.460 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/238 2024-04-02T17:01:49.460 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/238 2024-04-02T17:01:49.460 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:49.462 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:49.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:49.748 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-04-02T17:01:49.933 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:49.936 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:49.937 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:49.937 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:49.938 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:50.159 INFO:teuthology.orchestra.run.smithi050.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-04-02T17:01:50.162 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/239 2024-04-02T17:01:50.163 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/239 2024-04-02T17:01:50.163 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:50.165 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:50.187 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:50.188 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-04-02T17:01:50.365 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:50.369 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:50.369 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:50.370 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:50.370 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:50.587 INFO:teuthology.orchestra.run.smithi050.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-04-02T17:01:50.591 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/240 2024-04-02T17:01:50.591 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/240 2024-04-02T17:01:50.591 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:01:50.593 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:01:50.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:01:50.616 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-04-02T17:01:50.792 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:01:50.796 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:01:50.796 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:01:50.797 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:01:50.798 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:02:14.791 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:02:14.795 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:02:14.829 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T17:02:14.830+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 262445) UID: 0 2024-04-02T17:02:14.829 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T17:02:14.830+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 262445) UID: 0 2024-04-02T17:02:14.844 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T17:02:14.844+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48780) UID: 0 2024-04-02T17:02:14.845 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T17:02:14.844+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48780) UID: 0 2024-04-02T17:02:14.845 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T17:02:14.844+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48780) UID: 0 2024-04-02T17:02:14.851 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T17:02:14.844+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48780) UID: 0 2024-04-02T17:03:03.643 INFO:teuthology.orchestra.run.smithi050.stdout:generic/241 73s 2024-04-02T17:03:03.660 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/241 2024-04-02T17:03:03.660 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:03.666 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:04.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:04.779 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-04-02T17:03:05.150 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:05.154 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:05.154 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:05.155 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:05.155 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:05.483 INFO:teuthology.orchestra.run.smithi050.stdout:generic/242 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:03:05.489 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/242 2024-04-02T17:03:05.490 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/242 2024-04-02T17:03:05.490 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:05.492 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:05.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:05.901 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-04-02T17:03:06.284 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:06.295 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:06.296 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:06.298 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:06.298 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:06.975 INFO:teuthology.orchestra.run.smithi050.stdout:generic/243 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:03:06.982 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/243 2024-04-02T17:03:06.982 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/243 2024-04-02T17:03:06.982 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:06.984 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:07.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:07.312 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-04-02T17:03:07.496 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:07.500 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:07.500 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:07.501 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:07.502 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:07.709 INFO:teuthology.orchestra.run.smithi050.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:03:07.713 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/244 2024-04-02T17:03:07.713 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/244 2024-04-02T17:03:07.713 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:07.729 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:07.729 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-04-02T17:03:07.908 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:07.911 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:07.912 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:07.912 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:08.185 INFO:teuthology.orchestra.run.smithi050.stdout:generic/245 1s 2024-04-02T17:03:08.190 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/245 2024-04-02T17:03:08.190 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:08.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:08.250 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-04-02T17:03:08.688 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:08.691 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:08.692 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:08.692 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:08.940 INFO:teuthology.orchestra.run.smithi050.stdout:generic/246 0s 2024-04-02T17:03:08.945 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/246 2024-04-02T17:03:08.945 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:09.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:09.018 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-04-02T17:03:09.404 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:09.407 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:09.408 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:09.409 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:09.409 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:17.218 INFO:teuthology.orchestra.run.smithi050.stdout:generic/247 8s 2024-04-02T17:03:17.224 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/247 2024-04-02T17:03:17.224 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:17.226 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:17.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:17.235 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-04-02T17:03:17.407 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:17.410 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:17.410 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:17.411 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:17.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:17.644 INFO:teuthology.orchestra.run.smithi050.stdout:generic/248 0s 2024-04-02T17:03:17.649 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/248 2024-04-02T17:03:17.649 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:17.653 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:17.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:17.671 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-04-02T17:03:17.854 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:17.858 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:17.858 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:17.859 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:17.859 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:18.286 INFO:teuthology.orchestra.run.smithi050.stdout:generic/249 1s 2024-04-02T17:03:18.291 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/249 2024-04-02T17:03:18.292 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:18.293 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:18.340 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:18.340 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-04-02T17:03:18.796 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:18.800 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:18.800 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:18.801 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:18.801 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:19.043 INFO:teuthology.orchestra.run.smithi050.stdout:generic/250 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:03:19.047 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/250 2024-04-02T17:03:19.047 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/250 2024-04-02T17:03:19.047 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:19.049 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:19.062 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:19.062 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-04-02T17:03:19.236 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:19.240 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:19.241 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:19.241 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:19.242 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:19.521 INFO:teuthology.orchestra.run.smithi050.stdout:generic/251 [not run] FITRIM not supported on /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:19.528 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/251 2024-04-02T17:03:19.528 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/251 2024-04-02T17:03:19.528 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:19.530 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:19.542 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:19.543 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-04-02T17:03:19.731 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:19.735 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:19.735 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:19.736 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:19.736 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:19.943 INFO:teuthology.orchestra.run.smithi050.stdout:generic/252 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:03:19.947 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/252 2024-04-02T17:03:19.948 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/252 2024-04-02T17:03:19.948 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:19.950 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:19.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:19.962 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-04-02T17:03:20.148 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:20.151 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:20.152 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:20.153 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:20.153 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:20.481 INFO:teuthology.orchestra.run.smithi050.stdout:generic/253 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:03:20.486 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/253 2024-04-02T17:03:20.486 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/253 2024-04-02T17:03:20.486 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:20.493 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:20.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:20.841 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-04-02T17:03:21.021 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:21.024 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:21.024 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:21.026 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:21.026 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:21.352 INFO:teuthology.orchestra.run.smithi050.stdout:generic/254 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:03:21.360 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/254 2024-04-02T17:03:21.360 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/254 2024-04-02T17:03:21.360 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:21.363 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:21.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:21.712 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-04-02T17:03:21.893 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:21.897 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:21.897 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:21.898 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:21.898 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:22.194 INFO:teuthology.orchestra.run.smithi050.stdout:generic/255 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:03:22.198 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/255 2024-04-02T17:03:22.198 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/255 2024-04-02T17:03:22.198 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:22.200 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:22.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:22.243 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-04-02T17:03:22.697 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:22.701 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:22.702 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:22.703 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:22.703 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:23.448 INFO:teuthology.orchestra.run.smithi050.stdout:generic/256 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:03:23.452 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/256 2024-04-02T17:03:23.452 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/256 2024-04-02T17:03:23.452 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:23.454 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:23.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:23.471 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-04-02T17:03:23.662 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:23.665 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:23.666 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:23.667 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:23.667 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:24.503 INFO:teuthology.orchestra.run.smithi050.stdout:generic/257 1s 2024-04-02T17:03:24.508 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/257 2024-04-02T17:03:24.509 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:24.511 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:24.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:24.587 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-04-02T17:03:25.123 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:25.127 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:25.127 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:25.128 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:25.128 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:25.322 INFO:teuthology.orchestra.run.smithi050.stdout:generic/258 [not run] ceph does not support negative timestamps 2024-04-02T17:03:25.325 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/258 2024-04-02T17:03:25.325 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/258 2024-04-02T17:03:25.325 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:25.328 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:25.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:25.340 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-04-02T17:03:25.513 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:25.516 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:25.517 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:25.518 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:25.518 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:25.839 INFO:teuthology.orchestra.run.smithi050.stdout:generic/259 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:03:25.846 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/259 2024-04-02T17:03:25.846 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/259 2024-04-02T17:03:25.846 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:25.849 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:26.178 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:26.179 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-04-02T17:03:26.364 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:26.367 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:26.368 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:26.368 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:26.369 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:26.644 INFO:teuthology.orchestra.run.smithi050.stdout:generic/260 [not run] FITRIM not supported on /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:26.652 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/260 2024-04-02T17:03:26.652 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/260 2024-04-02T17:03:26.653 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:26.655 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:26.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:26.678 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-04-02T17:03:26.850 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:26.854 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:26.854 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:26.855 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:26.855 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:27.190 INFO:teuthology.orchestra.run.smithi050.stdout:generic/261 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:03:27.200 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/261 2024-04-02T17:03:27.200 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/261 2024-04-02T17:03:27.200 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:27.202 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:27.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:27.527 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-04-02T17:03:27.709 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:27.713 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:27.715 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:27.715 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:27.715 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:28.049 INFO:teuthology.orchestra.run.smithi050.stdout:generic/262 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:03:28.054 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/262 2024-04-02T17:03:28.054 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/262 2024-04-02T17:03:28.054 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:03:28.058 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:03:28.397 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:03:28.397 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-04-02T17:03:28.576 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:03:28.580 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:03:28.580 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:03:28.581 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:03:28.581 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:08.799 INFO:teuthology.orchestra.run.smithi050.stdout:generic/263 40s 2024-04-02T17:04:08.804 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/263 2024-04-02T17:04:08.804 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:08.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:08.854 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-04-02T17:04:09.260 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:09.264 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:09.264 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:09.265 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:09.265 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:09.623 INFO:teuthology.orchestra.run.smithi050.stdout:generic/264 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:09.630 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/264 2024-04-02T17:04:09.630 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/264 2024-04-02T17:04:09.630 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:09.632 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:09.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:09.994 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-04-02T17:04:10.185 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:10.188 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:10.189 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:10.190 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:10.190 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:10.587 INFO:teuthology.orchestra.run.smithi050.stdout:generic/265 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:10.594 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/265 2024-04-02T17:04:10.594 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/265 2024-04-02T17:04:10.594 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:10.596 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:10.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:10.909 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-04-02T17:04:11.107 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:11.110 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:11.111 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:11.112 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:11.112 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:11.502 INFO:teuthology.orchestra.run.smithi050.stdout:generic/266 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/266 2024-04-02T17:04:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/266 2024-04-02T17:04:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:11.510 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:11.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:11.636 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-04-02T17:04:11.968 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:11.971 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:11.972 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:11.972 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:11.972 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:12.354 INFO:teuthology.orchestra.run.smithi050.stdout:generic/267 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:12.365 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/267 2024-04-02T17:04:12.365 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/267 2024-04-02T17:04:12.365 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:12.367 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:12.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:12.688 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-04-02T17:04:12.872 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:12.875 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:12.875 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:12.876 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:12.876 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:13.271 INFO:teuthology.orchestra.run.smithi050.stdout:generic/268 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:13.278 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/268 2024-04-02T17:04:13.278 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/268 2024-04-02T17:04:13.279 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:13.280 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:13.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:13.400 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-04-02T17:04:13.658 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:13.662 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:13.662 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:13.663 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:13.663 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:13.885 INFO:teuthology.orchestra.run.smithi050.stdout:generic/269 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:04:13.890 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/269 2024-04-02T17:04:13.890 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/269 2024-04-02T17:04:13.890 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:13.891 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:13.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:13.904 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-04-02T17:04:14.082 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:14.085 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:14.086 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:14.087 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:14.087 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:14.284 INFO:teuthology.orchestra.run.smithi050.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:04:14.288 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/270 2024-04-02T17:04:14.288 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/270 2024-04-02T17:04:14.288 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:14.290 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:14.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:14.315 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-04-02T17:04:14.489 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:14.492 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:14.493 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:14.494 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:14.494 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:14.865 INFO:teuthology.orchestra.run.smithi050.stdout:generic/271 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:14.870 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/271 2024-04-02T17:04:14.870 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/271 2024-04-02T17:04:14.871 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:14.880 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:15.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:15.088 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-04-02T17:04:15.286 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:15.289 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:15.290 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:15.290 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:15.291 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:15.669 INFO:teuthology.orchestra.run.smithi050.stdout:generic/272 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:15.675 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/272 2024-04-02T17:04:15.675 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/272 2024-04-02T17:04:15.675 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:15.678 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:15.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:15.882 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-04-02T17:04:16.105 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:16.109 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:16.109 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:16.110 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:16.110 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:16.314 INFO:teuthology.orchestra.run.smithi050.stdout:generic/273 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:04:16.318 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/273 2024-04-02T17:04:16.318 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/273 2024-04-02T17:04:16.318 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:16.320 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:16.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:16.345 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-04-02T17:04:16.519 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:16.522 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:16.522 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:16.524 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:16.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:16.737 INFO:teuthology.orchestra.run.smithi050.stdout:generic/274 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T17:04:16.741 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/274 2024-04-02T17:04:16.741 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/274 2024-04-02T17:04:16.741 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:16.743 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:16.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:16.759 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-04-02T17:04:16.937 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:16.940 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:16.941 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:16.942 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:16.942 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:17.152 INFO:teuthology.orchestra.run.smithi050.stdout:generic/275 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:04:17.156 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/275 2024-04-02T17:04:17.156 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/275 2024-04-02T17:04:17.156 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:17.158 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:17.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:17.171 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-04-02T17:04:17.350 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:17.354 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:17.354 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:17.355 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:17.355 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:17.726 INFO:teuthology.orchestra.run.smithi050.stdout:generic/276 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:17.733 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/276 2024-04-02T17:04:17.733 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/276 2024-04-02T17:04:17.733 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:17.735 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:17.925 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:17.925 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-04-02T17:04:18.168 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:18.171 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:18.171 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:18.172 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:18.173 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:18.400 INFO:teuthology.orchestra.run.smithi050.stdout:generic/277 [not run] file system doesn't support chattr +A 2024-04-02T17:04:18.403 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/277 2024-04-02T17:04:18.404 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/277 2024-04-02T17:04:18.404 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:18.406 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:18.425 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:18.425 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-04-02T17:04:18.598 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:18.601 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:18.602 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:18.603 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:18.603 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:18.984 INFO:teuthology.orchestra.run.smithi050.stdout:generic/278 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:18.991 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/278 2024-04-02T17:04:18.991 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/278 2024-04-02T17:04:18.991 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:18.993 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:19.178 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:19.178 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-04-02T17:04:19.415 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:19.419 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:19.419 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:19.420 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:19.420 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:19.833 INFO:teuthology.orchestra.run.smithi050.stdout:generic/279 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:19.839 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/279 2024-04-02T17:04:19.840 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/279 2024-04-02T17:04:19.840 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:19.842 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:20.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:20.117 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-04-02T17:04:20.300 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:20.303 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:20.304 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:20.305 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:20.305 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:20.537 INFO:teuthology.orchestra.run.smithi050.stdout:generic/280 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:04:20.541 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/280 2024-04-02T17:04:20.541 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/280 2024-04-02T17:04:20.541 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:20.543 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:20.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:20.558 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-04-02T17:04:20.746 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:20.750 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:20.750 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:20.751 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:20.751 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:21.117 INFO:teuthology.orchestra.run.smithi050.stdout:generic/281 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/281 2024-04-02T17:04:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/281 2024-04-02T17:04:21.125 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:21.127 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:21.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:21.461 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-04-02T17:04:21.642 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:21.645 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:21.646 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:21.646 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:21.647 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:22.022 INFO:teuthology.orchestra.run.smithi050.stdout:generic/282 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:22.029 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/282 2024-04-02T17:04:22.029 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/282 2024-04-02T17:04:22.029 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:22.031 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:22.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:22.356 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-04-02T17:04:22.539 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:22.542 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:22.543 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:22.544 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:22.544 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:22.924 INFO:teuthology.orchestra.run.smithi050.stdout:generic/283 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:22.930 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/283 2024-04-02T17:04:22.930 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/283 2024-04-02T17:04:22.931 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:22.933 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:23.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:23.148 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-04-02T17:04:23.389 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:23.392 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:23.393 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:23.394 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:23.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:23.753 INFO:teuthology.orchestra.run.smithi050.stdout:generic/284 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:23.758 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/284 2024-04-02T17:04:23.758 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/284 2024-04-02T17:04:23.758 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:23.767 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:23.899 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:23.899 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-04-02T17:04:24.203 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:24.207 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:24.207 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:24.208 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:24.208 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:25.404 INFO:teuthology.orchestra.run.smithi050.stdout:generic/285 1s 2024-04-02T17:04:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/285 2024-04-02T17:04:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:25.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:25.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:25.811 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-04-02T17:04:26.192 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:26.196 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:26.196 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:26.197 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:26.197 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:26.414 INFO:teuthology.orchestra.run.smithi050.stdout:generic/286 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:04:26.417 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/286 2024-04-02T17:04:26.418 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/286 2024-04-02T17:04:26.418 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:26.420 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:26.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:26.437 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-04-02T17:04:26.625 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:26.629 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:26.629 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:26.630 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:26.630 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:27.118 INFO:teuthology.orchestra.run.smithi050.stdout:generic/287 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:27.125 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/287 2024-04-02T17:04:27.125 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/287 2024-04-02T17:04:27.125 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:27.127 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:27.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:27.337 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-04-02T17:04:27.650 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:27.655 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:27.655 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:27.657 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:27.657 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:28.086 INFO:teuthology.orchestra.run.smithi050.stdout:generic/288 [not run] FITRIM not supported on /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:28.111 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/288 2024-04-02T17:04:28.111 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/288 2024-04-02T17:04:28.111 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:28.130 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:28.130 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-04-02T17:04:28.393 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:28.397 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:28.398 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:28.399 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:28.399 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:28.895 INFO:teuthology.orchestra.run.smithi050.stdout:generic/289 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:28.902 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/289 2024-04-02T17:04:28.902 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/289 2024-04-02T17:04:28.902 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:28.905 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:29.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:29.150 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-04-02T17:04:29.428 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:29.432 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:29.433 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:29.434 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:29.434 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:29.932 INFO:teuthology.orchestra.run.smithi050.stdout:generic/290 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:29.938 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/290 2024-04-02T17:04:29.938 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/290 2024-04-02T17:04:29.938 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:29.941 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:30.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:30.234 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-04-02T17:04:30.505 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:30.509 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:30.510 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:30.512 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:30.512 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:30.998 INFO:teuthology.orchestra.run.smithi050.stdout:generic/291 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/291 2024-04-02T17:04:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/291 2024-04-02T17:04:31.004 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:31.007 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:31.177 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:31.177 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-04-02T17:04:31.477 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:31.481 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:31.481 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:31.482 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:31.482 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:31.982 INFO:teuthology.orchestra.run.smithi050.stdout:generic/292 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:31.989 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/292 2024-04-02T17:04:31.989 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/292 2024-04-02T17:04:31.989 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:31.991 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:32.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:32.264 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-04-02T17:04:32.516 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:32.520 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:32.521 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:32.522 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:32.522 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:33.040 INFO:teuthology.orchestra.run.smithi050.stdout:generic/293 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:33.045 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/293 2024-04-02T17:04:33.046 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/293 2024-04-02T17:04:33.046 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:33.049 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:33.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:33.195 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-04-02T17:04:33.486 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:33.489 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:33.490 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:33.491 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:33.491 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:33.944 INFO:teuthology.orchestra.run.smithi050.stdout:generic/294 0s 2024-04-02T17:04:33.951 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/294 2024-04-02T17:04:33.951 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:33.954 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:33.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:33.967 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-04-02T17:04:34.360 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:34.365 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:34.365 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:34.366 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:34.366 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:34.734 INFO:teuthology.orchestra.run.smithi050.stdout:generic/295 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:34.739 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/295 2024-04-02T17:04:34.739 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/295 2024-04-02T17:04:34.739 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:34.743 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:35.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:35.007 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-04-02T17:04:35.232 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:35.235 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:35.236 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:35.237 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:35.237 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:35.623 INFO:teuthology.orchestra.run.smithi050.stdout:generic/296 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:35.630 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/296 2024-04-02T17:04:35.630 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/296 2024-04-02T17:04:35.630 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:35.631 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:35.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:35.824 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-04-02T17:04:36.065 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:36.069 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:36.069 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:36.070 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:36.070 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:36.432 INFO:teuthology.orchestra.run.smithi050.stdout:generic/297 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:36.439 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/297 2024-04-02T17:04:36.439 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/297 2024-04-02T17:04:36.439 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:36.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:36.658 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-04-02T17:04:36.920 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:36.923 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:36.923 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:36.924 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:36.924 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:37.283 INFO:teuthology.orchestra.run.smithi050.stdout:generic/298 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:37.290 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/298 2024-04-02T17:04:37.290 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/298 2024-04-02T17:04:37.290 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:37.298 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:37.542 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:37.543 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-04-02T17:04:37.742 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:37.745 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:37.746 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:37.747 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:37.747 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:38.016 INFO:teuthology.orchestra.run.smithi050.stdout:generic/299 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:04:38.020 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/299 2024-04-02T17:04:38.020 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/299 2024-04-02T17:04:38.020 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:38.022 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:38.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:38.056 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-04-02T17:04:38.236 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:38.240 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:38.240 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:38.241 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:38.241 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:38.500 INFO:teuthology.orchestra.run.smithi050.stdout:generic/300 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:04:38.504 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/300 2024-04-02T17:04:38.504 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/300 2024-04-02T17:04:38.504 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:38.506 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:38.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:38.524 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-04-02T17:04:38.705 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:38.709 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:38.709 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:38.710 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:38.710 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:39.062 INFO:teuthology.orchestra.run.smithi050.stdout:generic/301 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:39.069 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/301 2024-04-02T17:04:39.069 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/301 2024-04-02T17:04:39.069 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:39.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:39.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:39.275 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-04-02T17:04:39.500 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:39.620 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:39.620 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:39.620 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:39.620 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:39.881 INFO:teuthology.orchestra.run.smithi050.stdout:generic/302 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:39.887 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/302 2024-04-02T17:04:39.887 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/302 2024-04-02T17:04:39.887 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:39.890 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:40.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:40.019 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-04-02T17:04:40.286 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:40.290 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:40.290 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:40.291 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:40.291 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:40.562 INFO:teuthology.orchestra.run.smithi050.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T17:04:40.565 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/303 2024-04-02T17:04:40.565 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/303 2024-04-02T17:04:40.565 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:40.567 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:40.620 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:40.621 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-04-02T17:04:41.022 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:41.026 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:41.026 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:41.027 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:41.027 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:41.278 INFO:teuthology.orchestra.run.smithi050.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T17:04:41.282 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/304 2024-04-02T17:04:41.282 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/304 2024-04-02T17:04:41.282 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:41.284 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:41.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:41.346 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-04-02T17:04:41.753 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:41.756 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:41.757 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:41.758 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:41.758 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:42.106 INFO:teuthology.orchestra.run.smithi050.stdout:generic/305 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:04:42.113 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/305 2024-04-02T17:04:42.113 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/305 2024-04-02T17:04:42.113 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:42.117 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:42.426 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:42.426 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-04-02T17:04:42.613 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:42.617 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:42.617 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:42.618 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:42.618 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:43.074 INFO:teuthology.orchestra.run.smithi050.stdout:generic/306 1s 2024-04-02T17:04:43.083 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/306 2024-04-02T17:04:43.083 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:43.085 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:43.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:43.420 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-04-02T17:04:43.883 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:43.887 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:43.887 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:43.888 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:43.888 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:45.364 INFO:teuthology.orchestra.run.smithi050.stdout:generic/307 2s 2024-04-02T17:04:45.370 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/307 2024-04-02T17:04:45.370 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:45.374 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:45.380 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:45.381 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-04-02T17:04:45.555 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:45.558 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:45.558 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:45.559 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:45.559 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:45.802 INFO:teuthology.orchestra.run.smithi050.stdout:generic/308 0s 2024-04-02T17:04:45.807 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/308 2024-04-02T17:04:45.807 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:45.809 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:45.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:45.827 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-04-02T17:04:46.003 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:46.006 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:46.007 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:46.007 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:46.008 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:47.276 INFO:teuthology.orchestra.run.smithi050.stdout:generic/309 1s 2024-04-02T17:04:47.281 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/309 2024-04-02T17:04:47.281 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:04:47.283 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:04:47.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:04:47.344 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-04-02T17:04:47.736 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:04:47.739 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:04:47.740 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:04:47.741 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:04:47.741 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:03.337 INFO:teuthology.orchestra.run.smithi050.stdout:generic/310 76s 2024-04-02T17:06:03.341 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/310 2024-04-02T17:06:03.342 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:03.344 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:03.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:03.428 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-04-02T17:06:03.761 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:03.766 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:03.766 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:03.767 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:03.767 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:04.049 INFO:teuthology.orchestra.run.smithi050.stdout:generic/311 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:06:04.052 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/311 2024-04-02T17:06:04.052 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/311 2024-04-02T17:06:04.052 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:04.054 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:04.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:04.071 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-04-02T17:06:04.244 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:04.248 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:04.248 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:04.249 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:04.249 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:04.451 INFO:teuthology.orchestra.run.smithi050.stdout:generic/312 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:06:04.454 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/312 2024-04-02T17:06:04.455 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/312 2024-04-02T17:06:04.455 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:04.457 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:04.473 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:04.473 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-04-02T17:06:04.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:06:04.596 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:06:04.646 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:04.650 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:04.650 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:04.651 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:04.651 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:08.910 INFO:teuthology.orchestra.run.smithi050.stdout:generic/313 4s 2024-04-02T17:06:08.916 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/313 2024-04-02T17:06:08.916 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:08.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:08.937 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-04-02T17:06:09.161 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:09.164 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:09.165 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:09.166 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:09.166 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:09.649 INFO:teuthology.orchestra.run.smithi050.stdout:generic/314 0s 2024-04-02T17:06:09.654 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/314 2024-04-02T17:06:09.654 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:09.656 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:09.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:09.671 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-04-02T17:06:09.853 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:09.857 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:09.857 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:09.857 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:09.857 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:10.067 INFO:teuthology.orchestra.run.smithi050.stdout:generic/315 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T17:06:10.070 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/315 2024-04-02T17:06:10.071 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/315 2024-04-02T17:06:10.071 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:10.073 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:10.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:10.087 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-04-02T17:06:10.264 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:10.267 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:10.268 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:10.269 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:10.269 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:10.592 INFO:teuthology.orchestra.run.smithi050.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-02T17:06:10.596 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/316 2024-04-02T17:06:10.596 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/316 2024-04-02T17:06:10.596 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:10.598 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:10.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:10.632 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-04-02T17:06:10.985 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:10.989 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:10.989 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:10.990 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:10.990 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:16.162 INFO:teuthology.orchestra.run.smithi050.stdout:generic/317 5s 2024-04-02T17:06:16.167 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/317 2024-04-02T17:06:16.167 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:16.170 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:16.179 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:16.179 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-04-02T17:06:16.367 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:16.370 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:16.371 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:16.372 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:16.372 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:16.866 INFO:teuthology.orchestra.run.smithi050.stdout:generic/318 0s 2024-04-02T17:06:16.875 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/318 2024-04-02T17:06:16.875 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:16.875 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:16.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:16.881 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-04-02T17:06:17.059 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:17.063 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:17.063 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:17.064 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:17.064 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:17.379 INFO:teuthology.orchestra.run.smithi050.stdout:generic/319 0s 2024-04-02T17:06:17.384 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/319 2024-04-02T17:06:17.384 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:17.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:17.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:17.395 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-04-02T17:06:17.574 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:17.577 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:17.578 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:17.579 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:17.579 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:17.783 INFO:teuthology.orchestra.run.smithi050.stdout:generic/320 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:06:17.786 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/320 2024-04-02T17:06:17.786 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/320 2024-04-02T17:06:17.787 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:17.789 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:17.801 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:17.801 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-04-02T17:06:17.974 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:17.977 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:17.978 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:17.979 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:17.979 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:18.181 INFO:teuthology.orchestra.run.smithi050.stdout:generic/321 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:06:18.185 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/321 2024-04-02T17:06:18.185 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/321 2024-04-02T17:06:18.185 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:18.187 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:18.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:18.199 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-04-02T17:06:18.374 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:18.377 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:18.378 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:18.379 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:18.379 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:18.581 INFO:teuthology.orchestra.run.smithi050.stdout:generic/322 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:06:18.586 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/322 2024-04-02T17:06:18.586 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/322 2024-04-02T17:06:18.586 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:06:18.588 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:18.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:06:18.600 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-04-02T17:06:18.777 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:06:18.780 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:06:18.781 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:06:18.782 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:06:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:06:34.731 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:06:34.733 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:07:04.900 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:07:04.903 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:07:04.934 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T17:07:04.935+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48840) UID: 0 2024-04-02T17:07:04.935 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T17:07:04.935+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48840) UID: 0 2024-04-02T17:07:04.935 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T17:07:04.935+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48840) UID: 0 2024-04-02T17:07:04.935 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T17:07:04.936+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48840) UID: 0 2024-04-02T17:08:19.429 INFO:teuthology.orchestra.run.smithi050.stdout:generic/323 121s 2024-04-02T17:08:19.433 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/323 2024-04-02T17:08:19.433 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:19.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:19.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:19.484 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-04-02T17:08:19.833 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:19.837 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:19.837 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:19.838 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:19.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:20.056 INFO:teuthology.orchestra.run.smithi050.stdout:generic/324 [not run] defragmentation not supported for fstype "ceph" 2024-04-02T17:08:20.059 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/324 2024-04-02T17:08:20.059 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/324 2024-04-02T17:08:20.059 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:20.062 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:20.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:20.073 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-04-02T17:08:20.247 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:20.250 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:20.251 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:20.252 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:20.252 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:20.457 INFO:teuthology.orchestra.run.smithi050.stdout:generic/325 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:08:20.461 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/325 2024-04-02T17:08:20.461 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/325 2024-04-02T17:08:20.461 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:20.463 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:20.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:20.475 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-04-02T17:08:20.643 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:20.646 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:20.647 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:20.647 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:20.648 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:20.995 INFO:teuthology.orchestra.run.smithi050.stdout:generic/326 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:21.000 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/326 2024-04-02T17:08:21.000 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/326 2024-04-02T17:08:21.000 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:21.006 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:21.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:21.089 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-04-02T17:08:21.372 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:21.376 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:21.376 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:21.377 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:21.377 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:21.703 INFO:teuthology.orchestra.run.smithi050.stdout:generic/327 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:21.709 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/327 2024-04-02T17:08:21.709 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/327 2024-04-02T17:08:21.709 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:21.712 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:21.786 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:21.786 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-04-02T17:08:22.095 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:22.099 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:22.099 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:22.100 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:22.100 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:22.437 INFO:teuthology.orchestra.run.smithi050.stdout:generic/328 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:22.444 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/328 2024-04-02T17:08:22.444 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/328 2024-04-02T17:08:22.444 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:22.445 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:22.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:22.517 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-04-02T17:08:22.825 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:22.829 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:22.829 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:22.830 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:22.830 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:23.200 INFO:teuthology.orchestra.run.smithi050.stdout:generic/329 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:23.206 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/329 2024-04-02T17:08:23.206 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/329 2024-04-02T17:08:23.206 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:23.210 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:23.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:23.361 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-04-02T17:08:23.586 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:23.590 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:23.590 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:23.591 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:23.591 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:23.921 INFO:teuthology.orchestra.run.smithi050.stdout:generic/330 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:23.928 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/330 2024-04-02T17:08:23.928 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/330 2024-04-02T17:08:23.928 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:23.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:24.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:24.028 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-04-02T17:08:24.309 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:24.313 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:24.313 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:24.314 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:24.314 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:24.685 INFO:teuthology.orchestra.run.smithi050.stdout:generic/331 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:24.692 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/331 2024-04-02T17:08:24.692 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/331 2024-04-02T17:08:24.692 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:24.694 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:24.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:24.788 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-04-02T17:08:25.063 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:25.067 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:25.068 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:25.068 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:25.068 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:25.414 INFO:teuthology.orchestra.run.smithi050.stdout:generic/332 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:25.419 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/332 2024-04-02T17:08:25.419 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/332 2024-04-02T17:08:25.419 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:25.422 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:25.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:25.527 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-04-02T17:08:25.799 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:25.802 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:25.803 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:25.804 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:25.804 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:26.140 INFO:teuthology.orchestra.run.smithi050.stdout:generic/333 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:26.145 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/333 2024-04-02T17:08:26.145 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/333 2024-04-02T17:08:26.145 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:26.149 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:26.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:26.238 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-04-02T17:08:26.512 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:26.516 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:26.516 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:26.517 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:26.518 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:26.848 INFO:teuthology.orchestra.run.smithi050.stdout:generic/334 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:08:26.854 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/334 2024-04-02T17:08:26.854 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/334 2024-04-02T17:08:26.854 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:26.857 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:27.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:27.073 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-04-02T17:08:27.291 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:27.295 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:27.295 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:27.296 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:27.296 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:27.502 INFO:teuthology.orchestra.run.smithi050.stdout:generic/335 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:08:27.506 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/335 2024-04-02T17:08:27.506 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/335 2024-04-02T17:08:27.506 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:27.508 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:27.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:27.520 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-04-02T17:08:27.693 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:27.697 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:27.697 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:27.698 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:27.698 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:27.926 INFO:teuthology.orchestra.run.smithi050.stdout:generic/336 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:08:29.369 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/336 2024-04-02T17:08:29.369 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/336 2024-04-02T17:08:29.369 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:29.369 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:29.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:29.369 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-04-02T17:08:29.566 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:29.568 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:29.569 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:29.570 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:29.570 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:29.916 INFO:teuthology.orchestra.run.smithi050.stdout:generic/337 0s 2024-04-02T17:08:29.922 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/337 2024-04-02T17:08:29.922 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:29.924 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:29.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:29.932 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-04-02T17:08:30.113 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:30.116 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:30.117 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:30.118 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:30.118 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:30.328 INFO:teuthology.orchestra.run.smithi050.stdout:generic/338 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:08:30.332 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/338 2024-04-02T17:08:30.332 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/338 2024-04-02T17:08:30.332 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:08:30.334 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:08:30.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:08:30.357 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-04-02T17:08:30.536 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:08:30.540 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:08:30.540 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:08:30.541 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:08:30.541 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:19.099 INFO:teuthology.orchestra.run.smithi050.stdout:generic/339 49s 2024-04-02T17:09:19.103 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/339 2024-04-02T17:09:19.103 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:19.106 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:19.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:19.115 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-04-02T17:09:19.287 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:19.290 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:19.290 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:19.291 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:19.291 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:21.574 INFO:teuthology.orchestra.run.smithi050.stdout:generic/340 2s 2024-04-02T17:09:21.588 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/340 2024-04-02T17:09:21.588 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:21.590 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:21.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:21.603 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-04-02T17:09:22.002 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:22.006 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:22.006 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:22.007 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:22.007 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:22.216 INFO:teuthology.orchestra.run.smithi050.stdout:generic/341 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:09:22.220 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/341 2024-04-02T17:09:22.220 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/341 2024-04-02T17:09:22.220 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:22.222 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:22.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:22.234 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-04-02T17:09:22.443 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:22.448 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:22.448 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:22.449 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:22.449 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:22.751 INFO:teuthology.orchestra.run.smithi050.stdout:generic/342 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:09:22.756 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/342 2024-04-02T17:09:22.756 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/342 2024-04-02T17:09:22.756 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:22.759 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:22.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:22.778 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-04-02T17:09:22.977 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:22.981 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:22.981 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:22.982 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:22.982 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:23.203 INFO:teuthology.orchestra.run.smithi050.stdout:generic/343 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:09:23.207 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/343 2024-04-02T17:09:23.207 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/343 2024-04-02T17:09:23.207 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:23.209 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:23.224 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:23.224 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-04-02T17:09:23.399 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:23.402 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:23.403 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:23.404 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:23.404 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:27.918 INFO:teuthology.orchestra.run.smithi050.stdout:generic/344 4s 2024-04-02T17:09:28.074 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/344 2024-04-02T17:09:28.074 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:28.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:28.088 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-04-02T17:09:28.285 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:28.289 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:28.289 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:28.290 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:28.290 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:32.965 INFO:teuthology.orchestra.run.smithi050.stdout:generic/345 4s 2024-04-02T17:09:32.978 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/345 2024-04-02T17:09:32.978 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:32.980 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:32.993 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:32.993 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-04-02T17:09:33.328 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:33.332 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:33.333 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:33.334 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:33.334 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:35.924 INFO:teuthology.orchestra.run.smithi050.stdout:generic/346 2s 2024-04-02T17:09:35.932 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/346 2024-04-02T17:09:35.932 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:35.934 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:35.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:35.951 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-04-02T17:09:36.308 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:36.312 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:36.312 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:36.313 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:36.313 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:36.623 INFO:teuthology.orchestra.run.smithi050.stdout:generic/347 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:09:36.627 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/347 2024-04-02T17:09:36.627 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/347 2024-04-02T17:09:36.627 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:36.630 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:36.650 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:36.650 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-04-02T17:09:36.879 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:36.891 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:36.892 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:36.896 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:36.896 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:37.120 INFO:teuthology.orchestra.run.smithi050.stdout:generic/348 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:09:37.124 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/348 2024-04-02T17:09:37.124 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/348 2024-04-02T17:09:37.124 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:37.126 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:37.141 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:37.141 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-04-02T17:09:37.321 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:37.324 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:37.325 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:37.326 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:37.326 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:37.616 INFO:teuthology.orchestra.run.smithi050.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-02T17:09:37.619 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/349 2024-04-02T17:09:37.619 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/349 2024-04-02T17:09:37.620 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:37.637 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:37.637 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-04-02T17:09:37.813 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:37.816 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:37.817 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:37.818 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:37.818 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:38.405 INFO:teuthology.orchestra.run.smithi050.stdout:generic/350 1s 2024-04-02T17:09:38.410 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/350 2024-04-02T17:09:38.411 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:38.413 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:38.421 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:38.421 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-04-02T17:09:38.600 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:38.604 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:38.604 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:38.605 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:38.605 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:38.828 INFO:teuthology.orchestra.run.smithi050.stdout:generic/351 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:09:38.832 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/351 2024-04-02T17:09:38.832 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/351 2024-04-02T17:09:38.832 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:38.834 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:38.849 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:38.849 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-04-02T17:09:39.021 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:39.024 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:39.025 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:39.026 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:39.026 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:39.342 INFO:teuthology.orchestra.run.smithi050.stdout:generic/352 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:09:39.348 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/352 2024-04-02T17:09:39.348 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/352 2024-04-02T17:09:39.348 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:39.353 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:39.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:39.782 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-04-02T17:09:39.957 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:39.961 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:39.961 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:39.962 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:39.962 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:40.307 INFO:teuthology.orchestra.run.smithi050.stdout:generic/353 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:09:40.318 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/353 2024-04-02T17:09:40.318 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/353 2024-04-02T17:09:40.318 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:40.324 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:40.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:40.689 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-04-02T17:09:40.869 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:40.873 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:40.874 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:40.874 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:40.875 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:44.816 INFO:teuthology.orchestra.run.smithi050.stdout:generic/354 4s 2024-04-02T17:09:44.822 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/354 2024-04-02T17:09:44.822 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:44.826 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:44.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:44.840 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-04-02T17:09:45.230 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:45.234 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:45.234 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:45.235 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:45.235 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:45.935 INFO:teuthology.orchestra.run.smithi050.stdout:generic/355 0s 2024-04-02T17:09:45.941 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/355 2024-04-02T17:09:45.941 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:45.944 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:45.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:45.996 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-04-02T17:09:46.465 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:46.469 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:46.469 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:46.471 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:46.471 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:46.905 INFO:teuthology.orchestra.run.smithi050.stdout:generic/356 [not run] swapfiles are not supported 2024-04-02T17:09:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/356 2024-04-02T17:09:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/356 2024-04-02T17:09:46.909 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:46.911 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:46.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:46.924 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-04-02T17:09:47.096 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:47.099 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:47.100 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:47.101 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:47.101 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:47.412 INFO:teuthology.orchestra.run.smithi050.stdout:generic/357 [not run] swapfiles are not supported 2024-04-02T17:09:47.416 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/357 2024-04-02T17:09:47.416 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/357 2024-04-02T17:09:47.416 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:47.418 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:47.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:47.431 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-04-02T17:09:47.610 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:47.613 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:47.614 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:47.614 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:47.615 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:47.945 INFO:teuthology.orchestra.run.smithi050.stdout:generic/358 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:09:47.951 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/358 2024-04-02T17:09:47.952 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/358 2024-04-02T17:09:47.952 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:47.954 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:48.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:48.273 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-04-02T17:09:48.453 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:48.457 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:48.457 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:48.458 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:48.458 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:48.821 INFO:teuthology.orchestra.run.smithi050.stdout:generic/359 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:09:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/359 2024-04-02T17:09:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/359 2024-04-02T17:09:48.827 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:48.830 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:49.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:49.148 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-04-02T17:09:49.335 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:49.338 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:49.339 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:49.340 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:49.340 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:49.571 INFO:teuthology.orchestra.run.smithi050.stdout:generic/360 0s 2024-04-02T17:09:49.576 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/360 2024-04-02T17:09:49.576 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:49.578 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:49.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:49.595 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-04-02T17:09:49.770 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:49.774 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:49.774 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:49.775 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:49.775 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:49.992 INFO:teuthology.orchestra.run.smithi050.stdout:generic/361 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:09:49.996 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/361 2024-04-02T17:09:49.997 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/361 2024-04-02T17:09:49.997 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:49.998 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:50.010 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:50.010 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-04-02T17:09:50.187 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:50.190 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:50.191 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:50.192 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:50.392 INFO:teuthology.orchestra.run.smithi050.stdout:generic/362 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:50.397 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/362 2024-04-02T17:09:50.397 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/362 2024-04-02T17:09:50.397 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:50.399 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:50.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:50.413 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-04-02T17:09:50.594 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:50.598 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:50.598 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:50.599 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:50.599 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:50.790 INFO:teuthology.orchestra.run.smithi050.stdout:generic/363 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:50.794 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/363 2024-04-02T17:09:50.794 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/363 2024-04-02T17:09:50.794 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:50.798 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:50.808 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:50.808 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-04-02T17:09:50.983 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:50.986 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:50.987 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:50.987 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:50.988 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:51.180 INFO:teuthology.orchestra.run.smithi050.stdout:generic/364 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:51.183 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/364 2024-04-02T17:09:51.184 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/364 2024-04-02T17:09:51.184 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:51.186 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:51.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:51.209 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-04-02T17:09:51.391 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:51.395 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:51.395 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:51.396 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:51.396 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:51.587 INFO:teuthology.orchestra.run.smithi050.stdout:generic/365 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:51.590 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/365 2024-04-02T17:09:51.591 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/365 2024-04-02T17:09:51.591 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:51.593 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:51.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:51.604 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-04-02T17:09:51.784 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:51.788 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:51.788 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:51.789 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:51.789 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:51.981 INFO:teuthology.orchestra.run.smithi050.stdout:generic/366 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:51.985 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/366 2024-04-02T17:09:51.985 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/366 2024-04-02T17:09:51.985 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:51.987 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:51.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:51.999 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-04-02T17:09:52.171 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:52.175 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:52.175 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:52.176 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:52.176 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:52.363 INFO:teuthology.orchestra.run.smithi050.stdout:generic/367 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:52.366 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/367 2024-04-02T17:09:52.366 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/367 2024-04-02T17:09:52.367 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:52.369 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:52.380 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:52.380 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-04-02T17:09:52.556 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:52.559 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:52.560 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:52.561 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:52.561 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:52.751 INFO:teuthology.orchestra.run.smithi050.stdout:generic/368 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:52.755 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/368 2024-04-02T17:09:52.755 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/368 2024-04-02T17:09:52.755 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:52.758 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:52.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:52.785 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-04-02T17:09:52.962 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:52.965 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:52.966 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:52.967 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:52.967 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:53.174 INFO:teuthology.orchestra.run.smithi050.stdout:generic/369 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:53.178 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/369 2024-04-02T17:09:53.178 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/369 2024-04-02T17:09:53.178 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:53.180 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:53.191 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:53.192 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-04-02T17:09:53.366 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:53.370 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:53.370 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:53.371 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:53.371 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:53.569 INFO:teuthology.orchestra.run.smithi050.stdout:generic/370 [not run] this test requires richacl support on $SCRATCH_DEV 2024-04-02T17:09:53.573 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/370 2024-04-02T17:09:53.573 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/370 2024-04-02T17:09:53.573 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:53.575 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:53.587 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:53.587 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-04-02T17:09:53.764 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:53.767 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:53.768 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:53.769 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:53.769 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:53.978 INFO:teuthology.orchestra.run.smithi050.stdout:generic/371 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:09:53.982 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/371 2024-04-02T17:09:53.982 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/371 2024-04-02T17:09:53.982 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:53.984 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:53.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:53.998 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-04-02T17:09:54.172 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:54.175 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:54.176 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:54.177 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:54.177 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:54.515 INFO:teuthology.orchestra.run.smithi050.stdout:generic/372 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:09:54.520 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/372 2024-04-02T17:09:54.520 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/372 2024-04-02T17:09:54.520 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:54.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:54.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:54.795 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-04-02T17:09:54.987 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:54.990 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:54.991 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:54.992 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:54.992 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:55.327 INFO:teuthology.orchestra.run.smithi050.stdout:generic/373 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:09:55.333 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/373 2024-04-02T17:09:55.334 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/373 2024-04-02T17:09:55.334 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:55.336 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:55.698 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:55.699 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-04-02T17:09:55.881 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:55.885 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:55.885 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:55.886 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:55.886 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:56.185 INFO:teuthology.orchestra.run.smithi050.stdout:generic/374 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-04-02T17:09:56.231 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/374 2024-04-02T17:09:56.231 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/374 2024-04-02T17:09:56.231 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:56.233 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:56.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:56.553 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-04-02T17:09:56.740 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:56.744 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:56.744 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:56.746 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:56.746 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:57.007 INFO:teuthology.orchestra.run.smithi050.stdout:generic/375 1s 2024-04-02T17:09:57.013 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/375 2024-04-02T17:09:57.013 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:57.015 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:57.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:57.060 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-04-02T17:09:57.504 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:57.508 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:57.508 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:57.509 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:57.509 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:57.717 INFO:teuthology.orchestra.run.smithi050.stdout:generic/376 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:09:57.721 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/376 2024-04-02T17:09:57.721 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/376 2024-04-02T17:09:57.721 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:57.723 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:57.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:57.735 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-04-02T17:09:57.919 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:57.922 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:57.923 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:57.924 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:57.924 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:58.235 INFO:teuthology.orchestra.run.smithi050.stdout:generic/377 0s 2024-04-02T17:09:58.240 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/377 2024-04-02T17:09:58.240 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:58.243 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:58.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:58.251 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-04-02T17:09:58.428 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:58.432 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:58.432 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:58.433 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:58.433 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:58.881 INFO:teuthology.orchestra.run.smithi050.stdout:generic/378 0s 2024-04-02T17:09:58.887 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/378 2024-04-02T17:09:58.887 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:58.889 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:58.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:58.949 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-04-02T17:09:59.345 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:59.349 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:59.350 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:59.351 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:59.351 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:59.560 INFO:teuthology.orchestra.run.smithi050.stdout:generic/379 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:09:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/379 2024-04-02T17:09:59.564 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/379 2024-04-02T17:09:59.564 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:59.566 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:59.577 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:59.578 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-04-02T17:09:59.753 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:09:59.756 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:09:59.757 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:09:59.758 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:09:59.758 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:59.952 INFO:teuthology.orchestra.run.smithi050.stdout:generic/380 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:09:59.955 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/380 2024-04-02T17:09:59.956 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/380 2024-04-02T17:09:59.956 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:09:59.958 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:09:59.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:09:59.969 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-04-02T17:10:00.147 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:00.150 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:00.151 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:00.152 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:00.347 INFO:teuthology.orchestra.run.smithi050.stdout:generic/381 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:10:00.351 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/381 2024-04-02T17:10:00.351 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/381 2024-04-02T17:10:00.351 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:00.353 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:00.384 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:00.384 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-04-02T17:10:00.566 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:00.569 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:00.570 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:00.571 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:00.571 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:00.773 INFO:teuthology.orchestra.run.smithi050.stdout:generic/382 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:10:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/382 2024-04-02T17:10:00.776 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/382 2024-04-02T17:10:00.777 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:00.779 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:00.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:00.790 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-04-02T17:10:00.975 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:00.979 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:00.979 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:00.980 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:00.980 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:01.176 INFO:teuthology.orchestra.run.smithi050.stdout:generic/383 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:10:01.179 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/383 2024-04-02T17:10:01.179 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/383 2024-04-02T17:10:01.179 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:01.182 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:01.193 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:01.194 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-04-02T17:10:01.366 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:01.370 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:01.370 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:01.371 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:01.371 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:01.574 INFO:teuthology.orchestra.run.smithi050.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:10:01.578 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/384 2024-04-02T17:10:01.578 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/384 2024-04-02T17:10:01.578 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:01.580 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:01.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:01.592 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-04-02T17:10:01.782 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:01.786 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:01.786 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:01.787 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:01.787 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:01.993 INFO:teuthology.orchestra.run.smithi050.stdout:generic/385 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:10:01.997 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/385 2024-04-02T17:10:01.997 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/385 2024-04-02T17:10:01.997 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:01.999 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:02.011 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:02.011 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-04-02T17:10:02.182 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:02.185 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:02.186 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:02.187 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:02.187 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:02.382 INFO:teuthology.orchestra.run.smithi050.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:10:02.385 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/386 2024-04-02T17:10:02.385 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/386 2024-04-02T17:10:02.386 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:02.388 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:02.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:02.400 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-04-02T17:10:02.571 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:02.575 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:02.575 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:02.576 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:02.576 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:02.929 INFO:teuthology.orchestra.run.smithi050.stdout:generic/387 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:10:02.936 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/387 2024-04-02T17:10:02.936 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/387 2024-04-02T17:10:02.936 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:02.939 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:03.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:03.315 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-04-02T17:10:03.484 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:03.487 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:03.488 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:03.489 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:03.489 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:03.698 INFO:teuthology.orchestra.run.smithi050.stdout:generic/388 [not run] require 172.21.15.50:6789:/scratch to be local device 2024-04-02T17:10:03.700 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/388 2024-04-02T17:10:03.700 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/388 2024-04-02T17:10:03.700 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:03.702 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:03.714 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:03.715 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-04-02T17:10:03.899 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:03.907 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:03.907 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:03.907 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:03.907 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:04.117 INFO:teuthology.orchestra.run.smithi050.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-04-02T17:10:04.120 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/389 2024-04-02T17:10:04.120 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/389 2024-04-02T17:10:04.121 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:04.123 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:04.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:04.135 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-04-02T17:10:04.316 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:04.320 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:04.320 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:04.321 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:04.557 INFO:teuthology.orchestra.run.smithi050.stdout:generic/390 [not run] ceph does not support freezing 2024-04-02T17:10:04.561 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/390 2024-04-02T17:10:04.561 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/390 2024-04-02T17:10:04.561 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:04.563 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:04.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:04.575 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-04-02T17:10:04.757 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:04.760 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:04.760 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:04.761 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:04.762 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:04.972 INFO:teuthology.orchestra.run.smithi050.stdout:generic/391 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:10:04.975 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/391 2024-04-02T17:10:04.975 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/391 2024-04-02T17:10:04.975 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:04.978 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:05.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:05.005 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-04-02T17:10:05.177 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:05.180 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:05.180 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:05.181 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:05.181 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:05.449 INFO:teuthology.orchestra.run.smithi050.stdout:generic/392 [not run] ceph does not support shutdown 2024-04-02T17:10:05.456 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/392 2024-04-02T17:10:05.456 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/392 2024-04-02T17:10:05.456 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:05.458 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:05.469 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:05.469 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-04-02T17:10:05.646 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:05.649 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:05.650 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:05.651 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:05.651 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:06.242 INFO:teuthology.orchestra.run.smithi050.stdout:generic/393 1s 2024-04-02T17:10:06.248 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/393 2024-04-02T17:10:06.248 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:06.250 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:06.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:06.259 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-04-02T17:10:06.437 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:06.441 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:06.441 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:06.442 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:06.442 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:06.678 INFO:teuthology.orchestra.run.smithi050.stdout:generic/394 0s 2024-04-02T17:10:06.683 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/394 2024-04-02T17:10:06.683 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:06.685 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:06.734 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:06.735 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-04-02T17:10:07.131 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:07.135 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:07.135 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:07.136 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:07.136 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:07.463 INFO:teuthology.orchestra.run.smithi050.stdout:generic/395 [not run] kernel does not support ceph encryption 2024-04-02T17:10:07.475 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/395 2024-04-02T17:10:07.475 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/395 2024-04-02T17:10:07.475 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:07.478 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:07.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:07.496 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-04-02T17:10:07.722 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:07.727 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:07.727 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:08.138 INFO:teuthology.orchestra.run.smithi050.stdout:generic/396 [not run] kernel does not support ceph encryption 2024-04-02T17:10:08.156 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/396 2024-04-02T17:10:08.156 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/396 2024-04-02T17:10:08.156 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:08.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:08.174 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-04-02T17:10:08.392 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:08.397 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:08.398 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:08.399 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:08.775 INFO:teuthology.orchestra.run.smithi050.stdout:generic/397 [not run] kernel does not support ceph encryption 2024-04-02T17:10:08.783 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/397 2024-04-02T17:10:08.783 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/397 2024-04-02T17:10:08.783 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:08.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:08.796 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-04-02T17:10:08.979 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:08.982 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:08.983 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:08.983 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:09.305 INFO:teuthology.orchestra.run.smithi050.stdout:generic/398 [not run] kernel does not support ceph encryption 2024-04-02T17:10:09.312 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/398 2024-04-02T17:10:09.312 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/398 2024-04-02T17:10:09.312 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:09.314 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:09.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:09.344 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-04-02T17:10:09.515 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:09.518 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:09.519 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:09.520 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:09.520 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:09.827 INFO:teuthology.orchestra.run.smithi050.stdout:generic/399 [not run] kernel does not support ceph encryption 2024-04-02T17:10:09.834 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/399 2024-04-02T17:10:09.835 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/399 2024-04-02T17:10:09.835 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:09.837 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:09.866 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:09.866 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-04-02T17:10:10.036 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:10.039 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:10.040 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:10.041 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:10.041 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:10.239 INFO:teuthology.orchestra.run.smithi050.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T17:10:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/400 2024-04-02T17:10:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/400 2024-04-02T17:10:10.243 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:10.245 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:10.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:10.267 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-04-02T17:10:10.444 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:10.448 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:10.448 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:10.449 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:10.449 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:10.837 INFO:teuthology.orchestra.run.smithi050.stdout:generic/401 0s 2024-04-02T17:10:10.851 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/401 2024-04-02T17:10:10.851 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:10.855 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:10.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:10.867 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-04-02T17:10:11.227 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:11.231 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:11.231 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:11.232 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:11.232 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:11.538 INFO:teuthology.orchestra.run.smithi050.stdout:generic/402 [not run] filesystem ceph timestamp bounds are unknown 2024-04-02T17:10:11.545 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/402 2024-04-02T17:10:11.545 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/402 2024-04-02T17:10:11.545 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:11.547 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:11.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:11.559 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-04-02T17:10:11.743 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:11.747 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:11.747 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:11.748 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:11.748 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:12.194 INFO:teuthology.orchestra.run.smithi050.stdout:generic/403 1s 2024-04-02T17:10:12.199 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/403 2024-04-02T17:10:12.199 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:12.201 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:12.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:12.210 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-04-02T17:10:12.387 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:12.391 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:12.391 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:12.392 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:12.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:12.604 INFO:teuthology.orchestra.run.smithi050.stdout:generic/404 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:10:12.608 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/404 2024-04-02T17:10:12.608 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/404 2024-04-02T17:10:12.608 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:12.610 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:12.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:12.625 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-04-02T17:10:12.798 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:12.801 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:12.802 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:12.803 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:12.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:13.031 INFO:teuthology.orchestra.run.smithi050.stdout:generic/405 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:10:13.035 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/405 2024-04-02T17:10:13.035 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/405 2024-04-02T17:10:13.035 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:13.037 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:13.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:13.049 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-04-02T17:10:13.221 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:13.224 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:13.225 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:13.226 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:13.226 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:14.695 INFO:teuthology.orchestra.run.smithi050.stdout:generic/406 1s 2024-04-02T17:10:14.700 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/406 2024-04-02T17:10:14.701 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:14.703 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:14.711 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:14.711 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-04-02T17:10:14.887 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:14.890 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:14.891 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:14.891 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:14.892 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:15.129 INFO:teuthology.orchestra.run.smithi050.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T17:10:15.132 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/407 2024-04-02T17:10:15.133 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/407 2024-04-02T17:10:15.133 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:15.135 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:15.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:15.203 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-04-02T17:10:15.600 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:15.610 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:15.610 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:15.612 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:15.612 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:15.849 INFO:teuthology.orchestra.run.smithi050.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T17:10:15.853 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/408 2024-04-02T17:10:15.853 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/408 2024-04-02T17:10:15.853 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:15.856 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:15.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:15.918 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-04-02T17:10:16.302 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:16.306 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:16.306 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:16.307 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:16.307 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:16.507 INFO:teuthology.orchestra.run.smithi050.stdout:generic/409 [not run] require 172.21.15.50:6789:/scratch to be local device 2024-04-02T17:10:16.510 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/409 2024-04-02T17:10:16.511 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/409 2024-04-02T17:10:16.511 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:16.513 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:16.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:16.524 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-04-02T17:10:16.698 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:16.701 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:16.702 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:16.703 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:16.703 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:16.905 INFO:teuthology.orchestra.run.smithi050.stdout:generic/410 [not run] require 172.21.15.50:6789:/scratch to be local device 2024-04-02T17:10:16.909 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/410 2024-04-02T17:10:16.909 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/410 2024-04-02T17:10:16.909 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:16.912 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:16.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:16.942 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-04-02T17:10:17.118 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:17.121 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:17.122 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:17.123 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:17.123 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:17.330 INFO:teuthology.orchestra.run.smithi050.stdout:generic/411 [not run] require 172.21.15.50:6789:/scratch to be local device 2024-04-02T17:10:17.334 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/411 2024-04-02T17:10:17.334 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/411 2024-04-02T17:10:17.334 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:17.337 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:17.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:17.348 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-04-02T17:10:17.525 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:17.528 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:17.528 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:17.530 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:17.530 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:17.956 INFO:teuthology.orchestra.run.smithi050.stdout:generic/412 0s 2024-04-02T17:10:17.962 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/412 2024-04-02T17:10:17.962 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:17.964 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:17.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:17.973 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-04-02T17:10:18.157 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:18.161 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:18.161 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:18.162 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:18.162 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:18.434 INFO:teuthology.orchestra.run.smithi050.stdout:generic/413 [not run] mount 172.21.15.50:6789:/scratch with dax failed 2024-04-02T17:10:18.438 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/413 2024-04-02T17:10:18.438 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/413 2024-04-02T17:10:18.438 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:18.440 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:18.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:18.463 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-04-02T17:10:18.638 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:18.642 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:18.642 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:18.643 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:18.643 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:18.964 INFO:teuthology.orchestra.run.smithi050.stdout:generic/414 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:10:18.971 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/414 2024-04-02T17:10:18.971 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/414 2024-04-02T17:10:18.971 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:18.973 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:19.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:19.268 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-04-02T17:10:19.454 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:19.458 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:19.458 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:19.460 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:19.460 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:19.825 INFO:teuthology.orchestra.run.smithi050.stdout:generic/415 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:10:19.830 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/415 2024-04-02T17:10:19.831 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/415 2024-04-02T17:10:19.831 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:19.837 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:20.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:20.160 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-04-02T17:10:20.357 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:20.360 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:20.361 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:20.362 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:20.362 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:20.566 INFO:teuthology.orchestra.run.smithi050.stdout:generic/416 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:10:20.570 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/416 2024-04-02T17:10:20.570 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/416 2024-04-02T17:10:20.570 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:20.572 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:20.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:20.584 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-04-02T17:10:20.767 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:20.770 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:20.770 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:20.772 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:20.772 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:21.044 INFO:teuthology.orchestra.run.smithi050.stdout:generic/417 [not run] ceph does not support shutdown 2024-04-02T17:10:21.052 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/417 2024-04-02T17:10:21.052 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/417 2024-04-02T17:10:21.052 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:21.054 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:21.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:21.078 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-04-02T17:10:21.269 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:21.272 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:21.273 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:21.274 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:21.274 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:21.519 INFO:teuthology.orchestra.run.smithi050.stdout:generic/418 [not run] require 172.21.15.50:6789:/test to be valid block disk 2024-04-02T17:10:21.523 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/418 2024-04-02T17:10:21.523 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/418 2024-04-02T17:10:21.523 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:21.525 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:21.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:21.540 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-04-02T17:10:21.719 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:21.723 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:21.723 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:21.725 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:21.726 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:22.032 INFO:teuthology.orchestra.run.smithi050.stdout:generic/419 [not run] kernel does not support ceph encryption 2024-04-02T17:10:22.040 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/419 2024-04-02T17:10:22.040 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/419 2024-04-02T17:10:22.040 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:22.042 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:22.066 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:22.066 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-04-02T17:10:22.252 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:22.276 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:22.276 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:22.276 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:22.276 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:22.578 INFO:teuthology.orchestra.run.smithi050.stdout:generic/420 0s 2024-04-02T17:10:22.583 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/420 2024-04-02T17:10:22.583 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:22.586 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:22.639 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:22.639 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-04-02T17:10:22.995 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:22.999 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:23.000 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:23.001 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:23.001 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:23.309 INFO:teuthology.orchestra.run.smithi050.stdout:generic/421 [not run] kernel does not support ceph encryption 2024-04-02T17:10:23.316 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/421 2024-04-02T17:10:23.316 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/421 2024-04-02T17:10:23.316 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:23.318 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:23.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:23.331 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-04-02T17:10:23.510 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:23.513 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:23.514 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:23.514 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:23.515 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:23.731 INFO:teuthology.orchestra.run.smithi050.stdout:generic/422 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T17:10:23.734 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/422 2024-04-02T17:10:23.735 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/422 2024-04-02T17:10:23.735 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:23.737 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:23.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:23.751 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-04-02T17:10:23.932 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:23.935 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:23.936 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:23.937 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:23.937 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:24.218 INFO:teuthology.orchestra.run.smithi050.stdout:generic/423 0s 2024-04-02T17:10:24.223 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/423 2024-04-02T17:10:24.223 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:24.225 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:24.283 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:24.283 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-04-02T17:10:24.660 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:24.664 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:24.664 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:24.665 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:24.665 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:24.874 INFO:teuthology.orchestra.run.smithi050.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2024-04-02T17:10:24.878 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/424 2024-04-02T17:10:24.878 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/424 2024-04-02T17:10:24.878 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:24.880 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:24.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:24.894 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-04-02T17:10:25.072 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:25.076 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:25.076 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:25.077 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:25.077 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:25.318 INFO:teuthology.orchestra.run.smithi050.stdout:generic/425 [not run] xfs_io fiemap -a failed (old kernel/wrong fs?) 2024-04-02T17:10:25.322 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/425 2024-04-02T17:10:25.322 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/425 2024-04-02T17:10:25.322 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:25.324 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:25.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:25.375 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-04-02T17:10:25.742 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:10:25.746 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:10:25.747 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:10:25.747 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:10:25.747 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:39.700 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:10:39.702 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:10:39.739 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T17:10:39.739+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48864) UID: 0 2024-04-02T17:10:39.739 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T17:10:39.739+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48864) UID: 0 2024-04-02T17:10:39.740 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T17:10:39.739+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48864) UID: 0 2024-04-02T17:10:39.740 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T17:10:39.739+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48864) UID: 0 2024-04-02T17:10:59.619 INFO:teuthology.orchestra.run.smithi050.stdout:generic/426 34s 2024-04-02T17:10:59.692 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/426 2024-04-02T17:10:59.693 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:10:59.695 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:10:59.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:10:59.872 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-04-02T17:11:00.940 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:00.943 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:00.943 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:00.944 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:00.945 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:01.266 INFO:teuthology.orchestra.run.smithi050.stdout:generic/427 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:11:01.270 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/427 2024-04-02T17:11:01.270 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/427 2024-04-02T17:11:01.270 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:01.272 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:01.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:01.288 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-04-02T17:11:01.461 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:01.465 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:01.465 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:01.466 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:01.466 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:01.704 INFO:teuthology.orchestra.run.smithi050.stdout:generic/428 0s 2024-04-02T17:11:01.709 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/428 2024-04-02T17:11:01.710 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:01.712 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:01.730 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:01.730 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-04-02T17:11:01.903 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:01.906 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:01.907 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:01.908 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:01.908 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:02.222 INFO:teuthology.orchestra.run.smithi050.stdout:generic/429 [not run] kernel does not support ceph encryption 2024-04-02T17:11:02.228 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/429 2024-04-02T17:11:02.228 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/429 2024-04-02T17:11:02.228 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:02.230 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:02.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:02.242 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-04-02T17:11:02.416 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:02.420 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:02.420 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:02.421 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:02.421 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:02.717 INFO:teuthology.orchestra.run.smithi050.stdout:generic/430 0s 2024-04-02T17:11:02.722 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/430 2024-04-02T17:11:02.722 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:02.724 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:02.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:02.805 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-04-02T17:11:03.154 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:03.157 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:03.158 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:03.159 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:03.159 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:03.448 INFO:teuthology.orchestra.run.smithi050.stdout:generic/431 0s 2024-04-02T17:11:03.461 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/431 2024-04-02T17:11:03.461 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:03.463 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:03.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:03.547 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-04-02T17:11:03.891 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:03.895 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:03.896 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:03.896 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:04.179 INFO:teuthology.orchestra.run.smithi050.stdout:generic/432 1s 2024-04-02T17:11:04.184 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/432 2024-04-02T17:11:04.184 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:04.186 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:04.238 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:04.238 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-04-02T17:11:04.600 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:04.607 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:04.607 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:04.608 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:04.608 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:04.889 INFO:teuthology.orchestra.run.smithi050.stdout:generic/433 0s 2024-04-02T17:11:04.894 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/433 2024-04-02T17:11:04.895 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:04.896 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:04.946 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:04.946 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-04-02T17:11:05.315 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:05.319 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:05.320 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:05.320 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:05.321 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:05.610 INFO:teuthology.orchestra.run.smithi050.stdout:generic/434 0s 2024-04-02T17:11:05.615 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/434 2024-04-02T17:11:05.615 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:05.618 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:05.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:05.669 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-04-02T17:11:06.039 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:06.043 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:06.044 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:06.045 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:06.045 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:06.362 INFO:teuthology.orchestra.run.smithi050.stdout:generic/435 [not run] kernel does not support ceph encryption 2024-04-02T17:11:06.369 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/435 2024-04-02T17:11:06.369 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/435 2024-04-02T17:11:06.369 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:06.371 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:06.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:06.383 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-04-02T17:11:06.562 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:06.565 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:06.566 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:06.567 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:06.567 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:07.175 INFO:teuthology.orchestra.run.smithi050.stdout:generic/436 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:11:07.179 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/436 2024-04-02T17:11:07.179 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/436 2024-04-02T17:11:07.179 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:07.181 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:07.217 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:07.217 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-04-02T17:11:07.582 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:07.586 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:07.587 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:07.588 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:07.588 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:08.468 INFO:teuthology.orchestra.run.smithi050.stdout:generic/437 1s 2024-04-02T17:11:08.473 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/437 2024-04-02T17:11:08.473 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:11:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:11:08.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:11:08.494 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-04-02T17:11:08.669 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:11:08.673 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:11:08.673 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:11:08.674 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:11:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:26.830 INFO:teuthology.orchestra.run.smithi050.stdout:generic/438 78s 2024-04-02T17:12:26.838 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/438 2024-04-02T17:12:26.838 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:26.838 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:26.855 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:26.856 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-04-02T17:12:27.032 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:27.036 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:27.036 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:27.037 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:27.037 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:27.955 INFO:teuthology.orchestra.run.smithi050.stdout:generic/439 0s 2024-04-02T17:12:27.960 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/439 2024-04-02T17:12:27.961 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:27.963 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:27.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:27.976 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-04-02T17:12:28.157 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:28.160 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:28.161 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:28.162 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:28.162 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:28.497 INFO:teuthology.orchestra.run.smithi050.stdout:generic/440 [not run] kernel does not support ceph encryption 2024-04-02T17:12:28.505 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/440 2024-04-02T17:12:28.506 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/440 2024-04-02T17:12:28.506 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:28.508 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:28.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:28.521 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-04-02T17:12:28.708 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:28.711 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:28.712 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:28.713 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:28.713 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:29.137 INFO:teuthology.orchestra.run.smithi050.stdout:generic/441 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:12:29.141 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/441 2024-04-02T17:12:29.141 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/441 2024-04-02T17:12:29.141 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:29.143 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:29.158 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:29.158 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-04-02T17:12:29.340 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:29.344 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:29.344 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:29.345 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:29.345 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:29.569 INFO:teuthology.orchestra.run.smithi050.stdout:generic/442 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:12:29.573 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/442 2024-04-02T17:12:29.573 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/442 2024-04-02T17:12:29.573 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:29.575 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:29.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:29.592 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-04-02T17:12:29.768 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:29.772 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:29.772 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:29.773 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:29.773 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:30.015 INFO:teuthology.orchestra.run.smithi050.stdout:generic/443 1s 2024-04-02T17:12:30.020 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/443 2024-04-02T17:12:30.020 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:30.022 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:30.040 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:30.041 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-04-02T17:12:30.224 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:30.228 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:30.228 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:30.229 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:30.229 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:30.511 INFO:teuthology.orchestra.run.smithi050.stdout:generic/444 0s 2024-04-02T17:12:30.516 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/444 2024-04-02T17:12:30.516 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:30.518 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:30.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:30.539 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-04-02T17:12:30.719 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:30.723 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:30.723 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:30.724 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:30.724 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:31.251 INFO:teuthology.orchestra.run.smithi050.stdout:generic/445 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:12:31.254 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/445 2024-04-02T17:12:31.255 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/445 2024-04-02T17:12:31.255 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:31.257 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:31.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:31.290 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-04-02T17:12:31.670 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:31.674 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:31.674 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:31.675 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:31.676 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:50.220 INFO:teuthology.orchestra.run.smithi050.stdout:generic/446 19s 2024-04-02T17:12:50.231 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/446 2024-04-02T17:12:50.231 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:50.235 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:50.246 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:50.246 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-04-02T17:12:50.583 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:50.587 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:50.587 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:50.588 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:50.588 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:50.954 INFO:teuthology.orchestra.run.smithi050.stdout:generic/447 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:12:50.961 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/447 2024-04-02T17:12:50.961 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/447 2024-04-02T17:12:50.961 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:50.964 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:51.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:51.278 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-04-02T17:12:51.456 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:51.459 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:51.460 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:51.461 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:51.461 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:52.429 INFO:teuthology.orchestra.run.smithi050.stdout:generic/448 1s 2024-04-02T17:12:52.434 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/448 2024-04-02T17:12:52.435 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:52.437 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:52.482 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:52.482 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-04-02T17:12:52.918 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:52.922 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:52.922 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:52.923 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:52.923 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:53.172 INFO:teuthology.orchestra.run.smithi050.stdout:generic/449 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T17:12:53.176 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/449 2024-04-02T17:12:53.176 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/449 2024-04-02T17:12:53.176 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:53.178 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:53.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:53.225 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-04-02T17:12:53.654 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:53.657 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:53.658 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:53.659 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:53.659 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:54.223 INFO:teuthology.orchestra.run.smithi050.stdout:generic/450 1s 2024-04-02T17:12:54.228 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/450 2024-04-02T17:12:54.228 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:12:54.231 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:12:54.283 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:12:54.283 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-04-02T17:12:54.642 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:12:54.650 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:12:54.651 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:12:54.652 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:12:54.652 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:24.964 INFO:teuthology.orchestra.run.smithi050.stdout:generic/451 30s 2024-04-02T17:13:24.969 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/451 2024-04-02T17:13:24.969 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:24.971 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:25.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:25.013 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-04-02T17:13:25.365 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:25.370 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:25.370 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:25.371 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:25.371 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:25.744 INFO:teuthology.orchestra.run.smithi050.stdout:generic/452 0s 2024-04-02T17:13:25.749 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/452 2024-04-02T17:13:25.749 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:25.752 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:25.760 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:25.760 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-04-02T17:13:25.937 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:25.941 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:25.941 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:25.942 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:25.942 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:26.150 INFO:teuthology.orchestra.run.smithi050.stdout:generic/453 [not run] ceph does not allow unrestricted byte streams for names 2024-04-02T17:13:26.153 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/453 2024-04-02T17:13:26.154 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/453 2024-04-02T17:13:26.154 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:26.156 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:26.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:26.169 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-04-02T17:13:26.347 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:26.350 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:26.351 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:26.352 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:26.352 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:26.572 INFO:teuthology.orchestra.run.smithi050.stdout:generic/454 [not run] ceph does not allow unrestricted byte streams for names 2024-04-02T17:13:26.576 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/454 2024-04-02T17:13:26.576 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/454 2024-04-02T17:13:26.576 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:26.578 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:26.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:26.599 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-04-02T17:13:26.784 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:26.787 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:26.788 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:26.789 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:26.789 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:27.058 INFO:teuthology.orchestra.run.smithi050.stdout:generic/455 [not run] This test requires a valid $LOGWRITES_DEV 2024-04-02T17:13:27.062 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/455 2024-04-02T17:13:27.062 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/455 2024-04-02T17:13:27.062 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:27.064 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:27.077 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:27.077 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-04-02T17:13:27.266 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:27.269 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:27.270 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:27.271 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:27.271 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:27.495 INFO:teuthology.orchestra.run.smithi050.stdout:generic/456 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T17:13:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/456 2024-04-02T17:13:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/456 2024-04-02T17:13:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:27.501 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:27.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:27.519 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-04-02T17:13:27.691 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:27.695 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:27.695 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:27.696 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:27.696 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:28.128 INFO:teuthology.orchestra.run.smithi050.stdout:generic/457 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:13:28.133 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/457 2024-04-02T17:13:28.133 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/457 2024-04-02T17:13:28.133 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:28.137 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:28.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:28.352 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-04-02T17:13:28.532 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:28.536 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:28.536 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:28.537 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:28.893 INFO:teuthology.orchestra.run.smithi050.stdout:generic/458 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T17:13:28.898 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/458 2024-04-02T17:13:28.901 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/458 2024-04-02T17:13:28.901 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:28.902 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:29.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:29.207 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-04-02T17:13:29.388 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:29.391 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:29.391 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:29.393 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:29.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:29.820 INFO:teuthology.orchestra.run.smithi050.stdout:generic/459 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:13:29.824 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/459 2024-04-02T17:13:29.824 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/459 2024-04-02T17:13:29.824 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:29.826 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:29.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:29.843 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-04-02T17:13:30.017 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:30.021 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:30.021 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:30.022 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:30.022 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:31.723 INFO:teuthology.orchestra.run.smithi050.stdout:generic/460 1s 2024-04-02T17:13:31.728 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/460 2024-04-02T17:13:31.728 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:31.730 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:31.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:31.742 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-04-02T17:13:31.917 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:31.921 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:31.921 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:31.922 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:31.922 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:32.035 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:13:32.038 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:13:32.119 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T17:13:32.119+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 420860) UID: 0 2024-04-02T17:13:32.119 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T17:13:32.119+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 420860) UID: 0 2024-04-02T17:13:32.119 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T17:13:32.119+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 420860) UID: 0 2024-04-02T17:13:32.119 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T17:13:32.119+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 420860) UID: 0 2024-04-02T17:13:32.222 INFO:teuthology.orchestra.run.smithi050.stdout:generic/461 [not run] ceph does not support shutdown 2024-04-02T17:13:32.226 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/461 2024-04-02T17:13:32.226 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/461 2024-04-02T17:13:32.226 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:32.228 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:32.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:32.242 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-04-02T17:13:32.421 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:32.424 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:32.424 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:32.425 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:32.425 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:32.766 INFO:teuthology.orchestra.run.smithi050.stdout:generic/462 [not run] mount 172.21.15.50:6789:/scratch with dax failed 2024-04-02T17:13:32.771 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/462 2024-04-02T17:13:32.771 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/462 2024-04-02T17:13:32.771 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:32.774 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:32.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:32.797 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-04-02T17:13:33.034 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:33.038 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:33.038 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:33.039 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:33.039 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:33.291 INFO:teuthology.orchestra.run.smithi050.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph 2024-04-02T17:13:33.295 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/463 2024-04-02T17:13:33.295 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/463 2024-04-02T17:13:33.295 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:13:33.297 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:13:33.353 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:13:33.353 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-04-02T17:13:33.758 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:13:33.762 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:13:33.762 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:13:33.763 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:13:33.763 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:29.590 INFO:teuthology.orchestra.run.smithi050.stdout:generic/464 56s 2024-04-02T17:14:29.595 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/464 2024-04-02T17:14:29.596 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:14:29.598 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:29.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:14:29.606 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-04-02T17:14:29.786 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:14:29.790 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:14:29.790 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:14:29.791 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:14:29.791 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:33.777 INFO:teuthology.orchestra.run.smithi050.stdout:generic/465 3s 2024-04-02T17:14:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/465 2024-04-02T17:14:33.790 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:14:33.794 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:33.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:14:33.805 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-04-02T17:14:34.235 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:14:34.239 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:14:34.239 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:14:34.240 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:14:34.240 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:34.442 INFO:teuthology.orchestra.run.smithi050.stdout:generic/466 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:14:34.446 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/466 2024-04-02T17:14:34.446 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/466 2024-04-02T17:14:34.446 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:14:34.448 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:34.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:14:34.462 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-04-02T17:14:34.641 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:14:34.645 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:14:34.645 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:14:34.646 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:14:34.646 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:45.157 INFO:teuthology.orchestra.run.smithi050.stdout:generic/467 10s 2024-04-02T17:14:45.270 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/467 2024-04-02T17:14:45.271 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:14:45.294 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:45.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:14:45.533 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-04-02T17:14:48.134 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:14:48.138 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:14:48.138 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:14:48.139 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:14:48.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:48.537 INFO:teuthology.orchestra.run.smithi050.stdout:generic/468 [not run] ceph does not support shutdown 2024-04-02T17:14:48.546 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/468 2024-04-02T17:14:48.546 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/468 2024-04-02T17:14:48.547 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:14:48.548 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:48.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:14:48.561 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-04-02T17:14:48.746 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:14:48.750 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:14:48.750 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:14:48.751 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:14:48.751 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:48.996 INFO:teuthology.orchestra.run.smithi050.stdout:generic/469 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T17:14:49.000 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/469 2024-04-02T17:14:49.000 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/469 2024-04-02T17:14:49.000 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:14:49.002 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:49.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:14:49.022 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-04-02T17:14:49.197 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:14:49.200 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:14:49.201 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:14:49.202 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:14:49.202 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:49.591 INFO:teuthology.orchestra.run.smithi050.stdout:generic/470 [not run] This test requires a valid $LOGWRITES_DEV 2024-04-02T17:14:49.595 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/470 2024-04-02T17:14:49.595 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/470 2024-04-02T17:14:49.595 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:14:49.597 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:14:49.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:14:49.611 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-04-02T17:14:49.796 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:14:49.800 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:14:49.800 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:14:49.801 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:14:49.801 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:37.374 INFO:teuthology.orchestra.run.smithi050.stdout:generic/471 48s 2024-04-02T17:15:37.382 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/471 2024-04-02T17:15:37.382 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:15:37.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:37.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:15:37.629 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-04-02T17:15:38.046 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:15:38.051 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:15:38.052 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:15:38.053 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:15:38.053 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:39.132 INFO:teuthology.orchestra.run.smithi050.stdout:generic/472 [not run] swapfiles are not supported 2024-04-02T17:15:39.136 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/472 2024-04-02T17:15:39.136 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/472 2024-04-02T17:15:39.136 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:15:39.138 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:39.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:15:39.153 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-04-02T17:15:39.340 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:15:39.344 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:15:39.344 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:15:39.345 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:15:39.345 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:39.604 INFO:teuthology.orchestra.run.smithi050.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-02T17:15:39.608 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/473 2024-04-02T17:15:39.608 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/473 2024-04-02T17:15:39.608 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:15:39.610 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:39.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:15:39.630 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-04-02T17:15:39.807 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:15:39.811 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:15:39.811 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:15:39.812 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:15:39.812 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:40.085 INFO:teuthology.orchestra.run.smithi050.stdout:generic/474 [not run] ceph does not support shutdown 2024-04-02T17:15:40.093 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/474 2024-04-02T17:15:40.093 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/474 2024-04-02T17:15:40.093 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:15:40.095 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:40.108 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:15:40.108 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-04-02T17:15:40.282 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:15:40.286 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:15:40.286 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:15:40.287 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:15:40.287 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:40.545 INFO:teuthology.orchestra.run.smithi050.stdout:generic/475 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T17:15:40.549 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/475 2024-04-02T17:15:40.549 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/475 2024-04-02T17:15:40.549 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T17:15:40.551 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:15:40.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T17:15:40.564 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-04-02T17:15:40.737 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T17:15:40.741 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T17:15:40.741 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T17:15:40.742 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T17:15:40.742 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T17:18:56.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:18:56.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:18:56.665 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T17:18:56.666+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 567707) UID: 0 2024-04-02T17:18:56.665 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T17:18:56.666+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 567707) UID: 0 2024-04-02T17:26:05.462 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T17:26:05.459+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 567838) UID: 0 2024-04-02T17:26:05.464 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T17:26:05.464+0000 7f1da7c55640 -1 Fail to open '/proc/567838/cmdline' error = (2) No such file or directory 2024-04-02T17:26:05.464 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T17:26:05.464+0000 7f1da7c55640 -1 received signal: Hangup from (PID: 567838) UID: 0 2024-04-02T17:26:05.468 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T17:26:05.460+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 567838) UID: 0 2024-04-02T17:26:05.472 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T17:26:05.471+0000 7f1ff3801640 -1 Fail to open '/proc/567838/cmdline' error = (2) No such file or directory 2024-04-02T17:26:05.472 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T17:26:05.471+0000 7f1ff3801640 -1 received signal: Hangup from (PID: 567838) UID: 0 2024-04-02T17:32:19.252 INFO:teuthology.orchestra.run.smithi050.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.0.log.1: 2024-04-02T17:32:19.253 INFO:teuthology.orchestra.run.smithi050.stderr:gzip: stdin: file size changed while zipping 2024-04-02T17:39:40.919 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:39:40.926 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T17:39:40.991 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T17:39:40.993+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 568047) UID: 0 2024-04-02T17:39:40.992 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T17:39:40.993+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 568047) UID: 0 2024-04-02T17:39:41.109 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T17:39:41.110+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49552) UID: 0 2024-04-02T17:39:41.110 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T17:39:41.111+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49552) UID: 0 2024-04-02T17:39:41.115 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T17:39:41.110+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49552) UID: 0 2024-04-02T17:39:41.121 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T17:39:41.122+0000 7f0530f8c640 -1 Fail to open '/proc/49552/cmdline' error = (2) No such file or directory 2024-04-02T17:39:41.121 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T17:39:41.122+0000 7f0530f8c640 -1 received signal: Hangup from (PID: 49552) UID: 0 2024-04-02T17:48:07.726 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T17:48:07.726+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568143) UID: 0 2024-04-02T17:48:07.727 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T17:48:07.727+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568143) UID: 0 2024-04-02T17:48:07.727 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T17:48:07.724+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568143) UID: 0 2024-04-02T17:48:07.728 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T17:48:07.726+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568143) UID: 0 2024-04-02T18:02:12.651 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T18:02:12.655 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T18:02:12.718 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T18:02:12.720+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 568290) UID: 0 2024-04-02T18:02:12.719 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T18:02:12.720+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 568290) UID: 0 2024-04-02T18:13:50.577 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T18:13:50.578+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568414) UID: 0 2024-04-02T18:13:50.591 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T18:13:50.578+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568414) UID: 0 2024-04-02T18:13:50.591 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T18:13:50.578+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568414) UID: 0 2024-04-02T18:13:50.591 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T18:13:50.578+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568414) UID: 0 2024-04-02T18:27:20.321 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T18:27:20.325 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T18:27:20.362 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T18:27:20.363+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 568581) UID: 0 2024-04-02T18:27:20.362 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T18:27:20.363+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 568581) UID: 0 2024-04-02T18:27:20.681 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T18:27:20.682+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50033) UID: 0 2024-04-02T18:27:20.687 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T18:27:20.683+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50033) UID: 0 2024-04-02T18:27:20.687 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T18:27:20.687+0000 7ff98796e640 -1 Fail to open '/proc/50033/cmdline' error = (2) No such file or directory 2024-04-02T18:27:20.687 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T18:27:20.687+0000 7ff98796e640 -1 received signal: Hangup from (PID: 50033) UID: 0 2024-04-02T18:27:20.696 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T18:27:20.682+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50033) UID: 0 2024-04-02T18:39:42.960 INFO:teuthology.orchestra.run.smithi050.stdout:generic/476 5042s 2024-04-02T18:39:42.991 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/476 2024-04-02T18:39:42.992 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:39:42.994 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:39:43.003 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:39:43.004 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-04-02T18:39:47.901 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:39:47.907 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:39:47.907 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:39:47.909 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:39:47.909 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:41:40.968 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T18:41:40.969+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568981) UID: 0 2024-04-02T18:41:40.969 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T18:41:40.968+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568981) UID: 0 2024-04-02T18:41:40.969 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T18:41:40.969+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568981) UID: 0 2024-04-02T18:41:40.970 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T18:41:40.969+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 568981) UID: 0 2024-04-02T18:46:58.323 INFO:teuthology.orchestra.run.smithi050.stdout:generic/477 28s 2024-04-02T18:46:58.509 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/477 2024-04-02T18:46:58.509 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:46:58.513 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:46:58.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:46:58.747 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-04-02T18:47:01.362 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:01.371 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:01.371 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:01.373 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:01.373 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:12.408 INFO:teuthology.orchestra.run.smithi050.stdout:generic/478 11s 2024-04-02T18:47:12.416 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/478 2024-04-02T18:47:12.416 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:12.419 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:12.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:12.450 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-04-02T18:47:12.741 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:12.747 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:12.747 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:12.749 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:12.749 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:13.256 INFO:teuthology.orchestra.run.smithi050.stdout:generic/479 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:13.261 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/479 2024-04-02T18:47:13.261 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/479 2024-04-02T18:47:13.261 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:13.264 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:13.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:13.295 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-04-02T18:47:13.559 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:13.563 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:13.564 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:13.565 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:13.565 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:13.919 INFO:teuthology.orchestra.run.smithi050.stdout:generic/480 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:13.925 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/480 2024-04-02T18:47:13.925 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/480 2024-04-02T18:47:13.925 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:13.927 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:13.957 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:13.958 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-04-02T18:47:14.237 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:14.245 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:14.246 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:14.247 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:14.247 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:14.567 INFO:teuthology.orchestra.run.smithi050.stdout:generic/481 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:14.572 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/481 2024-04-02T18:47:14.572 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/481 2024-04-02T18:47:14.572 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:14.575 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:14.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:14.598 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-04-02T18:47:14.883 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:14.888 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:14.888 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:14.890 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:14.890 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:15.264 INFO:teuthology.orchestra.run.smithi050.stdout:generic/482 [not run] This test requires a valid $LOGWRITES_DEV 2024-04-02T18:47:15.270 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/482 2024-04-02T18:47:15.270 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/482 2024-04-02T18:47:15.270 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:15.274 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:15.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:15.297 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-04-02T18:47:15.577 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:15.582 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:15.583 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:15.584 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:15.584 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:15.923 INFO:teuthology.orchestra.run.smithi050.stdout:generic/483 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:15.927 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/483 2024-04-02T18:47:15.927 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/483 2024-04-02T18:47:15.927 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:15.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:15.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:15.952 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-04-02T18:47:16.226 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:16.231 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:16.232 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:16.233 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:16.233 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:16.561 INFO:teuthology.orchestra.run.smithi050.stdout:generic/484 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:16.566 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/484 2024-04-02T18:47:16.566 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/484 2024-04-02T18:47:16.566 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:16.568 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:16.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:16.592 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-04-02T18:47:16.870 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:16.876 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:16.876 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:16.878 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:16.878 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:17.189 INFO:teuthology.orchestra.run.smithi050.stdout:generic/485 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T18:47:17.194 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/485 2024-04-02T18:47:17.195 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/485 2024-04-02T18:47:17.195 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:17.199 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:17.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:17.231 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-04-02T18:47:17.493 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:17.498 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:17.502 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:17.504 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:17.504 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:18.566 INFO:teuthology.orchestra.run.smithi050.stdout:generic/486 1s 2024-04-02T18:47:18.573 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/486 2024-04-02T18:47:18.573 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:18.576 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:18.588 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:18.588 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-04-02T18:47:18.870 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:18.876 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:18.877 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:18.879 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:18.879 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:19.239 INFO:teuthology.orchestra.run.smithi050.stdout:generic/487 [not run] This test requires a valid $SCRATCH_LOGDEV 2024-04-02T18:47:19.244 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/487 2024-04-02T18:47:19.245 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/487 2024-04-02T18:47:19.245 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:19.248 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:19.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:19.270 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-04-02T18:47:19.526 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:19.531 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:19.531 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:19.536 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:19.536 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:19.876 INFO:teuthology.orchestra.run.smithi050.stdout:generic/488 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-04-02T18:47:19.881 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/488 2024-04-02T18:47:19.881 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/488 2024-04-02T18:47:19.881 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:19.884 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:19.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:19.918 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-04-02T18:47:20.200 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:20.205 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:20.205 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:20.207 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:20.207 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:20.521 INFO:teuthology.orchestra.run.smithi050.stdout:generic/489 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:20.526 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/489 2024-04-02T18:47:20.526 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/489 2024-04-02T18:47:20.527 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:20.529 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:20.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:20.558 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-04-02T18:47:20.828 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:20.833 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:20.833 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:20.835 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:20.835 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:22.770 INFO:teuthology.orchestra.run.smithi050.stdout:generic/490 2s 2024-04-02T18:47:22.778 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/490 2024-04-02T18:47:22.778 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:22.781 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:22.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:22.883 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-04-02T18:47:23.425 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:23.432 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:23.433 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:23.434 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:23.434 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:24.164 INFO:teuthology.orchestra.run.smithi050.stdout:generic/491 [not run] ceph does not support freezing 2024-04-02T18:47:24.169 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/491 2024-04-02T18:47:24.169 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/491 2024-04-02T18:47:24.169 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:24.172 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:24.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:24.207 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-04-02T18:47:24.447 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:24.451 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:24.451 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:24.452 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:24.452 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:24.885 INFO:teuthology.orchestra.run.smithi050.stdout:generic/492 [not run] xfs_io label failed (old kernel/wrong fs?) 2024-04-02T18:47:24.890 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/492 2024-04-02T18:47:24.890 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/492 2024-04-02T18:47:24.890 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:24.893 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:24.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:24.923 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-04-02T18:47:25.196 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:25.202 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:25.203 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:25.205 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:25.205 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:25.837 INFO:teuthology.orchestra.run.smithi050.stdout:generic/493 [not run] swapfiles are not supported 2024-04-02T18:47:25.841 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/493 2024-04-02T18:47:25.841 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/493 2024-04-02T18:47:25.842 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:25.844 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:25.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:25.876 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-04-02T18:47:26.161 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:26.168 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:26.169 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:26.170 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:26.170 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:26.664 INFO:teuthology.orchestra.run.smithi050.stdout:generic/494 [not run] swapfiles are not supported 2024-04-02T18:47:26.668 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/494 2024-04-02T18:47:26.668 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/494 2024-04-02T18:47:26.668 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:26.671 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:26.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:26.700 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-04-02T18:47:26.978 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:26.982 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:26.983 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:26.984 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:26.984 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:27.510 INFO:teuthology.orchestra.run.smithi050.stdout:generic/495 [not run] swapfiles are not supported 2024-04-02T18:47:27.515 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/495 2024-04-02T18:47:27.515 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/495 2024-04-02T18:47:27.515 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:27.518 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:27.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:27.544 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-04-02T18:47:27.834 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:27.839 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:27.840 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:27.841 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:27.841 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:28.362 INFO:teuthology.orchestra.run.smithi050.stdout:generic/496 [not run] swapfiles are not supported 2024-04-02T18:47:28.367 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/496 2024-04-02T18:47:28.367 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/496 2024-04-02T18:47:28.367 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:28.370 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:28.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:28.399 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-04-02T18:47:28.675 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:28.679 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:28.679 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:28.681 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:28.681 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:29.168 INFO:teuthology.orchestra.run.smithi050.stdout:generic/497 [not run] swapfiles are not supported 2024-04-02T18:47:29.173 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/497 2024-04-02T18:47:29.174 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/497 2024-04-02T18:47:29.174 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:29.177 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:29.208 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:29.209 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-04-02T18:47:29.504 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:29.509 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:29.510 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:29.512 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:29.512 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:29.852 INFO:teuthology.orchestra.run.smithi050.stdout:generic/498 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:29.857 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/498 2024-04-02T18:47:29.857 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/498 2024-04-02T18:47:29.858 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:29.861 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:29.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:29.895 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-04-02T18:47:30.174 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:30.306 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:30.306 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:30.306 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:30.306 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:30.520 INFO:teuthology.orchestra.run.smithi050.stdout:generic/499 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T18:47:30.524 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/499 2024-04-02T18:47:30.525 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/499 2024-04-02T18:47:30.525 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:30.531 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:30.562 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:30.562 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-04-02T18:47:30.815 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:30.820 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:30.820 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:30.821 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:30.822 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:31.168 INFO:teuthology.orchestra.run.smithi050.stdout:generic/500 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:31.173 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/500 2024-04-02T18:47:31.173 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/500 2024-04-02T18:47:31.174 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:31.176 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:31.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:31.207 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-04-02T18:47:31.462 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:31.467 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:31.468 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:31.469 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:31.469 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:32.014 INFO:teuthology.orchestra.run.smithi050.stdout:generic/501 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T18:47:32.021 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/501 2024-04-02T18:47:32.021 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/501 2024-04-02T18:47:32.021 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:32.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:32.311 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-04-02T18:47:32.627 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:32.633 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:32.633 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:32.635 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:32.635 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:33.004 INFO:teuthology.orchestra.run.smithi050.stdout:generic/502 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:33.008 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/502 2024-04-02T18:47:33.008 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/502 2024-04-02T18:47:33.008 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:33.012 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:33.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:33.044 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-04-02T18:47:33.312 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:33.318 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:33.318 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:33.319 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:33.319 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:33.640 INFO:teuthology.orchestra.run.smithi050.stdout:generic/503 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T18:47:33.645 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/503 2024-04-02T18:47:33.645 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/503 2024-04-02T18:47:33.645 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:33.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:33.679 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-04-02T18:47:33.950 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:33.954 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:33.955 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:33.956 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:33.956 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:34.342 INFO:teuthology.orchestra.run.smithi050.stdout:generic/504 0s 2024-04-02T18:47:34.350 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/504 2024-04-02T18:47:34.350 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:34.353 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:34.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:34.392 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-04-02T18:47:34.656 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:34.660 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:34.661 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:34.661 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:34.662 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:35.085 INFO:teuthology.orchestra.run.smithi050.stdout:generic/505 [not run] ceph does not support shutdown 2024-04-02T18:47:35.111 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/505 2024-04-02T18:47:35.111 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/505 2024-04-02T18:47:35.111 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:35.114 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:35.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:35.136 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-04-02T18:47:35.396 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:35.402 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:35.402 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:35.403 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:35.403 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:35.703 INFO:teuthology.orchestra.run.smithi050.stdout:generic/506 [not run] disk quotas not supported by this filesystem type: ceph 2024-04-02T18:47:35.707 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/506 2024-04-02T18:47:35.707 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/506 2024-04-02T18:47:35.707 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:35.710 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:35.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:35.731 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-04-02T18:47:36.002 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:36.009 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:36.010 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:36.010 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:36.373 INFO:teuthology.orchestra.run.smithi050.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-04-02T18:47:36.378 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/507 2024-04-02T18:47:36.378 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/507 2024-04-02T18:47:36.379 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:36.383 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:36.414 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:36.414 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-04-02T18:47:36.719 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:36.724 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:36.725 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:36.726 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:36.726 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:37.050 INFO:teuthology.orchestra.run.smithi050.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph 2024-04-02T18:47:37.056 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/508 2024-04-02T18:47:37.056 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/508 2024-04-02T18:47:37.056 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:37.058 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:37.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:37.078 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-04-02T18:47:37.348 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:37.353 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:37.354 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:37.355 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:37.355 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:37.682 INFO:teuthology.orchestra.run.smithi050.stdout:generic/509 [not run] O_TMPFILE is not supported 2024-04-02T18:47:37.687 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/509 2024-04-02T18:47:37.687 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/509 2024-04-02T18:47:37.687 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:37.690 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:37.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:37.717 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-04-02T18:47:37.978 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:37.984 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:37.984 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:37.985 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:37.985 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:38.313 INFO:teuthology.orchestra.run.smithi050.stdout:generic/510 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:38.318 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/510 2024-04-02T18:47:38.318 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/510 2024-04-02T18:47:38.318 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:38.320 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:38.352 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:38.352 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-04-02T18:47:38.611 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:38.616 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:38.617 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:38.618 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:38.618 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:38.942 INFO:teuthology.orchestra.run.smithi050.stdout:generic/511 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-04-02T18:47:38.949 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/511 2024-04-02T18:47:38.949 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/511 2024-04-02T18:47:38.949 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:38.951 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:38.981 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:38.981 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-04-02T18:47:39.249 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:39.254 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:39.254 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:39.255 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:39.255 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:39.575 INFO:teuthology.orchestra.run.smithi050.stdout:generic/512 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T18:47:39.580 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/512 2024-04-02T18:47:39.580 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/512 2024-04-02T18:47:39.580 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:39.583 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:39.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:39.616 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-04-02T18:47:39.864 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:39.868 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:39.868 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:39.869 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:39.870 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:40.366 INFO:teuthology.orchestra.run.smithi050.stdout:generic/513 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T18:47:40.373 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/513 2024-04-02T18:47:40.373 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/513 2024-04-02T18:47:40.373 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:40.375 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:40.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:40.674 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-04-02T18:47:40.990 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:40.996 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:40.996 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:40.997 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:40.997 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:41.545 INFO:teuthology.orchestra.run.smithi050.stdout:generic/514 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T18:47:41.558 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/514 2024-04-02T18:47:41.558 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/514 2024-04-02T18:47:41.558 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:41.561 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:41.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:41.827 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-04-02T18:47:42.165 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:42.170 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:42.170 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:42.171 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:42.171 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:42.689 INFO:teuthology.orchestra.run.smithi050.stdout:generic/515 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T18:47:42.696 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/515 2024-04-02T18:47:42.696 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/515 2024-04-02T18:47:42.696 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:42.698 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:42.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:42.940 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-04-02T18:47:43.273 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:43.278 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:43.279 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:43.279 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:43.279 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:43.661 INFO:teuthology.orchestra.run.smithi050.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph 2024-04-02T18:47:43.667 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/516 2024-04-02T18:47:43.667 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/516 2024-04-02T18:47:43.667 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:43.671 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:43.738 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:43.738 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-04-02T18:47:44.254 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:44.259 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:44.260 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:44.263 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:44.263 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:44.749 INFO:teuthology.orchestra.run.smithi050.stdout:generic/517 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-04-02T18:47:44.806 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/517 2024-04-02T18:47:44.807 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/517 2024-04-02T18:47:44.807 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:44.808 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:44.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:44.961 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-04-02T18:47:45.314 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:45.320 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:45.321 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:45.322 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:45.322 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:46.102 INFO:teuthology.orchestra.run.smithi050.stdout:generic/518 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T18:47:46.108 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/518 2024-04-02T18:47:46.109 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/518 2024-04-02T18:47:46.109 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:46.113 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:46.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:46.382 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-04-02T18:47:46.657 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:46.662 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:46.662 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:46.663 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:46.664 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:47.147 INFO:teuthology.orchestra.run.smithi050.stdout:generic/519 [not run] FIBMAP not supported by this filesystem 2024-04-02T18:47:47.151 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/519 2024-04-02T18:47:47.151 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/519 2024-04-02T18:47:47.151 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:47.153 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:47.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:47.213 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-04-02T18:47:47.648 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:47.653 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:47.654 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:47.655 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:47.655 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:48.110 INFO:teuthology.orchestra.run.smithi050.stdout:generic/520 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T18:47:48.114 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/520 2024-04-02T18:47:48.115 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/520 2024-04-02T18:47:48.115 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T18:47:48.117 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:47:48.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T18:47:48.138 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-04-02T18:47:48.379 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T18:47:48.383 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T18:47:48.383 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T18:47:48.385 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T18:47:48.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T18:55:34.187 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T18:55:34.191 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T18:55:34.238 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T18:55:34.239+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50153) UID: 0 2024-04-02T18:55:34.238 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T18:55:34.239+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50153) UID: 0 2024-04-02T18:55:34.239 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T18:55:34.239+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50153) UID: 0 2024-04-02T18:55:34.240 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T18:55:34.240+0000 7f9ffa892640 -1 Fail to open '/proc/50153/cmdline' error = (2) No such file or directory 2024-04-02T18:55:34.240 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T18:55:34.240+0000 7f9ffa892640 -1 received signal: Hangup from (PID: 50153) UID: 0 2024-04-02T18:55:34.318 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T18:55:34.319+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 590452) UID: 0 2024-04-02T18:55:34.319 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T18:55:34.319+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 590452) UID: 0 2024-04-02T19:06:31.100 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T19:06:31.100+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590474) UID: 0 2024-04-02T19:06:31.125 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T19:06:31.100+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590474) UID: 0 2024-04-02T19:06:31.126 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T19:06:31.100+0000 7f1da7c55640 -1 Fail to open '/proc/590474/cmdline' error = (2) No such file or directory 2024-04-02T19:06:31.126 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T19:06:31.100+0000 7f1da7c55640 -1 received signal: Hangup from (PID: 590474) UID: 0 2024-04-02T19:06:31.126 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T19:06:31.100+0000 7fde4bd4f640 -1 Fail to open '/proc/590474/cmdline' error = (2) No such file or directory 2024-04-02T19:06:31.127 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T19:06:31.100+0000 7fde4bd4f640 -1 received signal: Hangup from (PID: 590474) UID: 0 2024-04-02T19:15:35.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:15:35.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:15:35.435 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T19:15:35.436+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590503) UID: 0 2024-04-02T19:15:35.435 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T19:15:35.436+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590503) UID: 0 2024-04-02T19:15:35.435 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T19:15:35.436+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590503) UID: 0 2024-04-02T19:15:35.436 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T19:15:35.436+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590503) UID: 0 2024-04-02T19:15:35.442 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T19:15:35.444+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50215) UID: 0 2024-04-02T19:15:35.443 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T19:15:35.444+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50215) UID: 0 2024-04-02T19:15:35.443 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T19:15:35.444+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50215) UID: 0 2024-04-02T19:15:35.443 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T19:15:35.444+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50215) UID: 0 2024-04-02T19:31:42.449 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:31:42.472 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:31:42.517 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T19:31:42.519+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50255) UID: 0 2024-04-02T19:31:42.518 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T19:31:42.519+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50255) UID: 0 2024-04-02T19:31:42.518 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T19:31:42.519+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50255) UID: 0 2024-04-02T19:31:42.518 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T19:31:42.519+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50255) UID: 0 2024-04-02T19:31:42.529 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T19:31:42.530+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590537) UID: 0 2024-04-02T19:31:42.529 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T19:31:42.530+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590537) UID: 0 2024-04-02T19:31:42.529 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T19:31:42.530+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590537) UID: 0 2024-04-02T19:31:42.530 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T19:31:42.530+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590537) UID: 0 2024-04-02T19:38:23.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:38:23.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:38:23.286 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T19:38:23.287+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50281) UID: 0 2024-04-02T19:38:23.287 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T19:38:23.287+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50281) UID: 0 2024-04-02T19:38:23.287 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T19:38:23.288+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50281) UID: 0 2024-04-02T19:38:23.287 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T19:38:23.288+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50281) UID: 0 2024-04-02T19:42:47.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:42:47.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:43:17.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:43:17.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:43:47.951 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:43:47.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:44:18.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:44:18.047 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:44:48.116 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:44:48.119 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:45:18.480 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:45:18.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:45:48.742 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:45:48.745 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:46:19.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:46:19.040 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:46:49.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:46:49.110 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:47:19.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:47:19.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:47:49.288 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:47:49.291 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:48:19.418 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:48:19.421 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:48:19.458 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T19:48:19.460+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590786) UID: 0 2024-04-02T19:48:19.459 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T19:48:19.460+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590786) UID: 0 2024-04-02T19:48:19.459 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T19:48:19.460+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590786) UID: 0 2024-04-02T19:48:19.459 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T19:48:19.460+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590786) UID: 0 2024-04-02T19:51:50.498 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:51:50.501 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:51:50.541 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T19:51:50.543+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590811) UID: 0 2024-04-02T19:51:50.542 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T19:51:50.543+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590811) UID: 0 2024-04-02T19:51:50.542 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T19:51:50.543+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590811) UID: 0 2024-04-02T19:51:50.542 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T19:51:50.543+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 590811) UID: 0 2024-04-02T19:59:03.888 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:59:03.891 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T19:59:03.933 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T19:59:03.934+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50550) UID: 0 2024-04-02T19:59:03.934 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T19:59:03.935+0000 7fd244e01640 -1 Fail to open '/proc/50550/cmdline' error = (2) No such file or directory 2024-04-02T19:59:03.934 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T19:59:03.935+0000 7fd244e01640 -1 received signal: Hangup from (PID: 50550) UID: 0 2024-04-02T19:59:03.934 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T19:59:03.934+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50550) UID: 0 2024-04-02T19:59:03.934 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T19:59:03.935+0000 7f0530f8c640 -1 Fail to open '/proc/50550/cmdline' error = (2) No such file or directory 2024-04-02T19:59:03.934 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T19:59:03.935+0000 7f0530f8c640 -1 received signal: Hangup from (PID: 50550) UID: 0 2024-04-02T20:10:36.870 INFO:teuthology.orchestra.run.smithi050.stdout:generic/521 4968s 2024-04-02T20:10:36.882 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/521 2024-04-02T20:10:36.883 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T20:10:36.886 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T20:10:36.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T20:10:36.969 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-04-02T20:10:37.456 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T20:10:37.461 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T20:10:37.461 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T20:10:37.462 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T20:10:37.463 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T20:15:45.938 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:15:45.941 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:15:45.982 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T20:15:45.984+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 591356) UID: 0 2024-04-02T20:15:45.982 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T20:15:45.984+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 591356) UID: 0 2024-04-02T20:19:17.842 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T20:19:17.843+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591361) UID: 0 2024-04-02T20:19:17.855 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T20:19:17.843+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591361) UID: 0 2024-04-02T20:19:17.855 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T20:19:17.843+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591361) UID: 0 2024-04-02T20:19:17.870 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T20:19:17.843+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591361) UID: 0 2024-04-02T20:23:54.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:23:54.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:23:54.238 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T20:23:54.239+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591388) UID: 0 2024-04-02T20:23:54.238 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T20:23:54.239+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591388) UID: 0 2024-04-02T20:23:54.239 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T20:23:54.239+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591388) UID: 0 2024-04-02T20:23:54.239 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T20:23:54.239+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591388) UID: 0 2024-04-02T20:23:54.245 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T20:23:54.246+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50623) UID: 0 2024-04-02T20:23:54.246 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T20:23:54.247+0000 7fd244e01640 -1 received signal: Hangup from (PID: 50623) UID: 0 2024-04-02T20:23:54.246 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T20:23:54.246+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50623) UID: 0 2024-04-02T20:23:54.247 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T20:23:54.246+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50623) UID: 0 2024-04-02T20:28:33.040 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:28:33.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:28:33.085 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T20:28:33.086+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50647) UID: 0 2024-04-02T20:28:33.085 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T20:28:33.086+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50647) UID: 0 2024-04-02T20:28:33.086 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T20:28:33.086+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50647) UID: 0 2024-04-02T20:28:33.087 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T20:28:33.086+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50647) UID: 0 2024-04-02T20:32:54.439 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:32:54.442 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:33:24.532 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:33:24.535 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:33:24.568 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T20:33:24.569+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591452) UID: 0 2024-04-02T20:33:24.569 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T20:33:24.569+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591452) UID: 0 2024-04-02T20:33:24.569 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T20:33:24.569+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591452) UID: 0 2024-04-02T20:33:24.569 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T20:33:24.569+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591452) UID: 0 2024-04-02T20:37:11.180 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:37:11.183 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:37:11.219 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T20:37:11.220+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50708) UID: 0 2024-04-02T20:37:11.219 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T20:37:11.220+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50708) UID: 0 2024-04-02T20:37:11.220 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T20:37:11.220+0000 7fd244e01640 -1 received signal: Hangup from (PID: 50708) UID: 0 2024-04-02T20:37:11.221 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T20:37:11.220+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50708) UID: 0 2024-04-02T20:37:11.224 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T20:37:11.225+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591473) UID: 0 2024-04-02T20:37:11.224 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T20:37:11.225+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591473) UID: 0 2024-04-02T20:37:11.225 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T20:37:11.225+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591473) UID: 0 2024-04-02T20:37:11.225 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T20:37:11.225+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591473) UID: 0 2024-04-02T20:41:29.775 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:41:29.779 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:41:29.818 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T20:41:29.819+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50733) UID: 0 2024-04-02T20:41:29.819 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T20:41:29.819+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50733) UID: 0 2024-04-02T20:41:29.819 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T20:41:29.819+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50733) UID: 0 2024-04-02T20:41:29.819 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T20:41:29.820+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50733) UID: 0 2024-04-02T20:45:53.763 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:45:53.767 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:45:53.808 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T20:45:53.809+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50756) UID: 0 2024-04-02T20:45:53.808 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T20:45:53.809+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50756) UID: 0 2024-04-02T20:45:53.808 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T20:45:53.809+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50756) UID: 0 2024-04-02T20:45:53.808 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T20:45:53.809+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50756) UID: 0 2024-04-02T20:50:16.180 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:50:16.184 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:50:46.531 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:50:46.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:51:16.868 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:51:16.871 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:51:47.121 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:51:47.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:52:17.462 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:52:17.465 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:52:47.780 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:52:47.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:53:18.104 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:53:18.108 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:53:48.394 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:53:48.397 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:54:18.712 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:54:18.715 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:54:48.793 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:54:48.797 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:55:18.914 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:55:18.917 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:55:48.990 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:55:48.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:55:49.029 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T20:55:49.030+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591742) UID: 0 2024-04-02T20:55:49.030 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T20:55:49.030+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591742) UID: 0 2024-04-02T20:55:49.030 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T20:55:49.030+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591742) UID: 0 2024-04-02T20:55:49.030 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T20:55:49.030+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591742) UID: 0 2024-04-02T20:59:31.501 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:59:31.504 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T20:59:31.545 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T20:59:31.545+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591767) UID: 0 2024-04-02T20:59:31.545 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T20:59:31.545+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591767) UID: 0 2024-04-02T20:59:31.545 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T20:59:31.546+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591767) UID: 0 2024-04-02T20:59:31.562 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T20:59:31.546+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 591767) UID: 0 2024-04-02T21:03:42.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:03:42.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:03:42.727 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T21:03:42.728+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51037) UID: 0 2024-04-02T21:03:42.727 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T21:03:42.728+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51037) UID: 0 2024-04-02T21:03:42.727 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T21:03:42.728+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51037) UID: 0 2024-04-02T21:03:42.728 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T21:03:42.728+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51037) UID: 0 2024-04-02T21:05:03.609 INFO:teuthology.orchestra.run.smithi050.stdout:generic/522 3266s 2024-04-02T21:05:03.630 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/522 2024-04-02T21:05:03.631 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:03.634 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:03.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:03.669 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-04-02T21:05:03.918 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:03.922 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:03.923 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:03.924 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:03.924 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:04.471 INFO:teuthology.orchestra.run.smithi050.stdout:generic/523 0s 2024-04-02T21:05:04.479 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/523 2024-04-02T21:05:04.480 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:04.482 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:04.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:04.493 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-04-02T21:05:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:04.745 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:04.746 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:04.747 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:04.747 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:09.142 INFO:teuthology.orchestra.run.smithi050.stdout:generic/524 5s 2024-04-02T21:05:09.149 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/524 2024-04-02T21:05:09.149 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:09.152 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:09.163 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:09.164 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-04-02T21:05:09.391 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:09.395 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:09.395 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:09.397 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:09.397 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:09.823 INFO:teuthology.orchestra.run.smithi050.stdout:generic/525 [not run] filesystem does not support huge file size 2024-04-02T21:05:09.837 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/525 2024-04-02T21:05:09.837 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/525 2024-04-02T21:05:09.837 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:09.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:09.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:09.872 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-04-02T21:05:10.100 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:10.105 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:10.105 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:10.106 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:10.106 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:10.478 INFO:teuthology.orchestra.run.smithi050.stdout:generic/526 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T21:05:10.483 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/526 2024-04-02T21:05:10.484 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/526 2024-04-02T21:05:10.484 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:10.487 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:10.518 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:10.518 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-04-02T21:05:10.743 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:10.749 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:10.749 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:10.751 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:10.751 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:11.090 INFO:teuthology.orchestra.run.smithi050.stdout:generic/527 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T21:05:11.096 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/527 2024-04-02T21:05:11.096 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/527 2024-04-02T21:05:11.096 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:11.099 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:11.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:11.135 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-04-02T21:05:11.377 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:11.381 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:11.382 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:11.386 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:11.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:generic/528 0s 2024-04-02T21:05:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/528 2024-04-02T21:05:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:11.777 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:11.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:11.821 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-04-02T21:05:12.250 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:12.255 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:12.255 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:12.256 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:12.256 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:12.760 INFO:teuthology.orchestra.run.smithi050.stdout:generic/529 0s 2024-04-02T21:05:12.771 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/529 2024-04-02T21:05:12.771 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:12.774 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:12.784 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:12.784 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-04-02T21:05:13.018 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:13.022 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:13.023 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:13.024 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:13.024 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:13.433 INFO:teuthology.orchestra.run.smithi050.stdout:generic/530 [not run] ceph does not support shutdown 2024-04-02T21:05:13.452 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/530 2024-04-02T21:05:13.452 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/530 2024-04-02T21:05:13.452 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:13.455 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:13.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:13.477 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-04-02T21:05:13.700 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:13.704 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:13.705 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:13.706 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:13.706 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:13.992 INFO:teuthology.orchestra.run.smithi050.stdout:generic/531 [not run] O_TMPFILE is not supported 2024-04-02T21:05:13.997 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/531 2024-04-02T21:05:13.997 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/531 2024-04-02T21:05:13.997 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:14.000 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:14.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:14.020 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-04-02T21:05:14.255 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:14.259 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:14.260 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:14.261 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:14.261 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:14.588 INFO:teuthology.orchestra.run.smithi050.stdout:generic/532 0s 2024-04-02T21:05:14.595 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/532 2024-04-02T21:05:14.595 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:14.598 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:14.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:14.616 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-04-02T21:05:14.869 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:14.874 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:14.875 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:14.876 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:14.876 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:15.821 INFO:teuthology.orchestra.run.smithi050.stdout:generic/533 0s 2024-04-02T21:05:15.832 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/533 2024-04-02T21:05:15.833 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:15.836 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:15.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:15.855 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-04-02T21:05:16.095 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:16.100 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:16.101 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:16.102 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:16.102 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:16.535 INFO:teuthology.orchestra.run.smithi050.stdout:generic/534 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T21:05:16.540 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/534 2024-04-02T21:05:16.540 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/534 2024-04-02T21:05:16.540 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:16.543 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:16.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:16.568 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-04-02T21:05:16.813 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:16.817 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:16.818 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:16.819 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:16.820 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:17.272 INFO:teuthology.orchestra.run.smithi050.stdout:generic/535 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T21:05:17.277 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/535 2024-04-02T21:05:17.277 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/535 2024-04-02T21:05:17.277 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:17.280 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:17.299 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:17.299 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-04-02T21:05:17.540 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:17.545 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:17.545 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:17.546 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:17.546 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:17.943 INFO:teuthology.orchestra.run.smithi050.stdout:generic/536 [not run] ceph does not support shutdown 2024-04-02T21:05:17.958 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/536 2024-04-02T21:05:17.958 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/536 2024-04-02T21:05:17.958 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:17.960 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:17.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:17.979 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-04-02T21:05:18.217 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:18.222 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:18.223 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:18.224 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:18.224 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:18.876 INFO:teuthology.orchestra.run.smithi050.stdout:generic/537 [not run] FSTRIM not supported 2024-04-02T21:05:18.898 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/537 2024-04-02T21:05:18.898 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/537 2024-04-02T21:05:18.898 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:05:18.902 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:05:18.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:05:18.921 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-04-02T21:05:19.155 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:05:19.166 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:05:19.166 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:05:19.167 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:05:19.167 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:07:52.041 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:07:52.044 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:07:52.082 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T21:07:52.083+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51645) UID: 0 2024-04-02T21:07:52.083 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T21:07:52.083+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51645) UID: 0 2024-04-02T21:07:52.083 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T21:07:52.083+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51645) UID: 0 2024-04-02T21:07:52.083 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T21:07:52.083+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51645) UID: 0 2024-04-02T21:12:08.933 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:12:08.937 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:12:08.979 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T21:12:08.981+0000 7ff98796e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51683) UID: 0 2024-04-02T21:12:08.980 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T21:12:08.981+0000 7fd244e01640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51683) UID: 0 2024-04-02T21:12:08.980 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T21:12:08.981+0000 7f0530f8c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51683) UID: 0 2024-04-02T21:12:08.980 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T21:12:08.981+0000 7f9ffa892640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51683) UID: 0 2024-04-02T21:12:08.984 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T21:12:08.985+0000 7fcf2c643640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 605698) UID: 0 2024-04-02T21:12:08.984 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T21:12:08.985+0000 7fe34ca6b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 605698) UID: 0 2024-04-02T21:13:03.542 INFO:teuthology.orchestra.run.smithi050.stdout:generic/538 464s 2024-04-02T21:13:03.550 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/538 2024-04-02T21:13:03.550 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:03.553 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:03.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:03.586 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2024-04-02T21:13:04.040 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:04.047 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:04.047 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:04.048 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:04.049 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:06.301 INFO:teuthology.orchestra.run.smithi050.stdout:generic/539 2s 2024-04-02T21:13:06.308 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/539 2024-04-02T21:13:06.308 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:06.311 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:06.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:06.375 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2024-04-02T21:13:06.798 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:06.805 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:06.807 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:06.811 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:06.811 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:07.388 INFO:teuthology.orchestra.run.smithi050.stdout:generic/540 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/540 2024-04-02T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/540 2024-04-02T21:13:07.394 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:07.398 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:07.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:07.656 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2024-04-02T21:13:08.003 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:08.009 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:08.010 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:08.011 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:08.546 INFO:teuthology.orchestra.run.smithi050.stdout:generic/541 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T21:13:08.555 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/541 2024-04-02T21:13:08.555 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/541 2024-04-02T21:13:08.555 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:08.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:08.773 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2024-04-02T21:13:09.104 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:09.111 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:09.111 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:09.112 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:09.112 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:09.667 INFO:teuthology.orchestra.run.smithi050.stdout:generic/542 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T21:13:09.672 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/542 2024-04-02T21:13:09.672 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/542 2024-04-02T21:13:09.672 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:09.676 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:09.884 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:09.884 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2024-04-02T21:13:10.252 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:10.257 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:10.258 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:10.259 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:10.259 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:10.799 INFO:teuthology.orchestra.run.smithi050.stdout:generic/543 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T21:13:10.804 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/543 2024-04-02T21:13:10.804 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/543 2024-04-02T21:13:10.804 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:10.808 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:11.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:11.088 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2024-04-02T21:13:11.378 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:11.382 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:11.383 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:11.384 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:11.384 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:11.898 INFO:teuthology.orchestra.run.smithi050.stdout:generic/544 [not run] Reflink not supported by scratch filesystem type: ceph 2024-04-02T21:13:11.905 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/544 2024-04-02T21:13:11.905 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/544 2024-04-02T21:13:11.905 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:11.912 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:12.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:12.131 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2024-04-02T21:13:12.473 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:12.479 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:12.480 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:12.481 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:12.481 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:12.894 INFO:teuthology.orchestra.run.smithi050.stdout:generic/545 [not run] file system doesn't support chattr +i 2024-04-02T21:13:12.899 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/545 2024-04-02T21:13:12.899 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/545 2024-04-02T21:13:12.899 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:12.902 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:12.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:12.936 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2024-04-02T21:13:13.206 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:13.213 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:13.213 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:13.214 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:13.214 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:13.602 INFO:teuthology.orchestra.run.smithi050.stdout:generic/546 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-04-02T21:13:13.607 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/546 2024-04-02T21:13:13.607 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/546 2024-04-02T21:13:13.608 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:13.617 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:13.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:13.651 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2024-04-02T21:13:13.902 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:13.906 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:13.907 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:13.908 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:13.908 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:14.516 INFO:teuthology.orchestra.run.smithi050.stdout:generic/547 [not run] require 172.21.15.50:6789:/scratch to be valid block disk 2024-04-02T21:13:14.521 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/547 2024-04-02T21:13:14.521 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/547 2024-04-02T21:13:14.521 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:14.530 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:14.555 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:14.555 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2024-04-02T21:13:14.794 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:14.798 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:14.799 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:14.800 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:14.800 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:15.584 INFO:teuthology.orchestra.run.smithi050.stdout:generic/548 [not run] kernel does not support ceph encryption 2024-04-02T21:13:15.600 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/548 2024-04-02T21:13:15.601 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/548 2024-04-02T21:13:15.601 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:15.604 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:15.620 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:15.620 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2024-04-02T21:13:15.851 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:15.855 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:15.856 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:15.857 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:15.857 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:16.496 INFO:teuthology.orchestra.run.smithi050.stdout:generic/549 [not run] kernel does not support ceph encryption 2024-04-02T21:13:16.521 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/549 2024-04-02T21:13:16.521 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/549 2024-04-02T21:13:16.521 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:16.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:16.542 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:16.542 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2024-04-02T21:13:16.779 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:16.784 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:16.784 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:16.785 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:16.785 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:17.343 INFO:teuthology.orchestra.run.smithi050.stdout:generic/550 [not run] kernel does not support ceph encryption 2024-04-02T21:13:17.358 INFO:teuthology.orchestra.run.smithi050.stdout:Ran: generic/550 2024-04-02T21:13:17.358 INFO:teuthology.orchestra.run.smithi050.stdout:Not run: generic/550 2024-04-02T21:13:17.358 INFO:teuthology.orchestra.run.smithi050.stdout:Passed all 1 tests 2024-04-02T21:13:17.361 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:13:17.381 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-02T21:13:17.381 DEBUG:teuthology.orchestra.run.smithi050:> (cd /tmp/tmp.CCuraAyba9xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2024-04-02T21:13:17.635 INFO:teuthology.orchestra.run.smithi050.stdout:FSTYP -- ceph 2024-04-02T21:13:17.641 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM -- Linux/x86_64 smithi050 5.14.0-434.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 26 19:20:47 UTC 2024 2024-04-02T21:13:17.641 INFO:teuthology.orchestra.run.smithi050.stdout:MKFS_OPTIONS -- 172.21.15.50:6789:/scratch 2024-04-02T21:13:17.642 INFO:teuthology.orchestra.run.smithi050.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBwMAxmW+97DBAAgL3ZUHCV80y9yx8ZUhNUZg== -o context=system_u:object_r:root_t:s0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch 2024-04-02T21:13:17.642 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:16:00.742 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T21:16:00.743+0000 7f2262b9a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 613094) UID: 0 2024-04-02T21:16:00.743 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T21:16:00.743+0000 7f1da7c55640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 613094) UID: 0 2024-04-02T21:16:00.743 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T21:16:00.743+0000 7f1ff3801640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 613094) UID: 0 2024-04-02T21:16:00.743 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T21:16:00.743+0000 7fde4bd4f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 613094) UID: 0 2024-04-02T21:16:28.167 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T21:16:28.165+0000 7f9ff0e9c640 -1 bdev(0x55ea8992ee00 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-04-02T21:16:28.167 INFO:tasks.ceph.osd.5.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f9ff0e9c640 time 2024-04-02T21:16:28.166785+0000 2024-04-02T21:16:28.167 INFO:tasks.ceph.osd.5.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:16:28.167 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.167 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x55ea8784cd8e] 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T21:16:28.166+0000 7f9ff0e9c640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f9ff0e9c640 time 2024-04-02T21:16:28.166785+0000 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x55ea8784cd8e] 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.168 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr:*** Caught signal (Aborted) ** 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: in thread 7f9ff0e9c640 thread_name:bstore_aio 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9ffdc54db0] 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f9ffdca154c] 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: raise() 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: abort() 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x55ea8784ce4f] 2024-04-02T21:16:28.169 INFO:tasks.ceph.osd.5.smithi188.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 7: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-02T21:16:28.167+0000 7f9ff0e9c640 -1 *** Caught signal (Aborted) ** 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: in thread 7f9ff0e9c640 thread_name:bstore_aio 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9ffdc54db0] 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f9ffdca154c] 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: raise() 2024-04-02T21:16:28.170 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: abort() 2024-04-02T21:16:28.171 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x55ea8784ce4f] 2024-04-02T21:16:28.171 INFO:tasks.ceph.osd.5.smithi188.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.171 INFO:tasks.ceph.osd.5.smithi188.stderr: 7: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.171 INFO:tasks.ceph.osd.5.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.171 INFO:tasks.ceph.osd.5.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.171 INFO:tasks.ceph.osd.5.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-02T21:16:28.171 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.176 INFO:tasks.ceph.osd.5.smithi188.stderr: -43> 2024-04-02T21:16:28.165+0000 7f9ff0e9c640 -1 bdev(0x55ea8992ee00 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-04-02T21:16:28.176 INFO:tasks.ceph.osd.5.smithi188.stderr: -1> 2024-04-02T21:16:28.166+0000 7f9ff0e9c640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f9ff0e9c640 time 2024-04-02T21:16:28.166785+0000 2024-04-02T21:16:28.176 INFO:tasks.ceph.osd.5.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:16:28.176 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.176 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x55ea8784cd8e] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 0> 2024-04-02T21:16:28.167+0000 7f9ff0e9c640 -1 *** Caught signal (Aborted) ** 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: in thread 7f9ff0e9c640 thread_name:bstore_aio 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9ffdc54db0] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f9ffdca154c] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: raise() 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: abort() 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x55ea8784ce4f] 2024-04-02T21:16:28.177 INFO:tasks.ceph.osd.5.smithi188.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.178 INFO:tasks.ceph.osd.5.smithi188.stderr: 7: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.178 INFO:tasks.ceph.osd.5.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.178 INFO:tasks.ceph.osd.5.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.178 INFO:tasks.ceph.osd.5.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-02T21:16:28.178 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: -88> 2024-04-02T21:16:28.165+0000 7f9ff0e9c640 -1 bdev(0x55ea8992ee00 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: -46> 2024-04-02T21:16:28.166+0000 7f9ff0e9c640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f9ff0e9c640 time 2024-04-02T21:16:28.166785+0000 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x55ea8784cd8e] 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.186 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: -45> 2024-04-02T21:16:28.167+0000 7f9ff0e9c640 -1 *** Caught signal (Aborted) ** 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: in thread 7f9ff0e9c640 thread_name:bstore_aio 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9ffdc54db0] 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f9ffdca154c] 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 3: raise() 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 4: abort() 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x55ea8784ce4f] 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x55ea882288f0] 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 7: ceph-osd(+0xdd1a61) [0x55ea88228a61] 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f9ffdc9f802] 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f9ffdc3f450] 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-02T21:16:28.187 INFO:tasks.ceph.osd.5.smithi188.stderr: 2024-04-02T21:16:28.460 INFO:tasks.ceph.osd.5.smithi188.stderr:daemon-helper: command failed with exit status 5 2024-04-02T21:16:35.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-02T21:16:41.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~7s 2024-04-02T21:16:48.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~13s 2024-04-02T21:16:54.968 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2024-04-02T21:17:01.582 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~26s 2024-04-02T21:17:08.196 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~33s 2024-04-02T21:17:14.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2024-04-02T21:17:21.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~46s 2024-04-02T21:17:28.040 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~53s 2024-04-02T21:17:34.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~60s 2024-04-02T21:17:41.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~66s 2024-04-02T21:17:47.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~73s 2024-04-02T21:17:54.495 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~79s 2024-04-02T21:18:01.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~86s 2024-04-02T21:18:07.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~93s 2024-04-02T21:18:14.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~99s 2024-04-02T21:18:20.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~106s 2024-04-02T21:18:27.563 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~112s 2024-04-02T21:18:34.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~119s 2024-04-02T21:18:40.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~126s 2024-04-02T21:18:47.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~132s 2024-04-02T21:18:54.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~139s 2024-04-02T21:19:00.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~146s 2024-04-02T21:19:07.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~152s 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fde42359640 time 2024-04-02T21:19:09.782414+0000 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555c62440d8e] 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.782 INFO:tasks.ceph.osd.2.smithi050.stderr:*** Caught signal (Aborted) ** 2024-04-02T21:19:09.783 INFO:tasks.ceph.osd.2.smithi050.stderr: in thread 7fde42359640 thread_name:bstore_aio 2024-04-02T21:19:09.783 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T21:19:09.782+0000 7fde42359640 -1 bdev(0x555c64650e00 /var/lib/ceph/osd/ceph-2/block) _aio_thread got r=-28 ((28) No space left on device) 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fde4f254db0] 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fde4f2a154c] 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: raise() 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: abort() 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555c62440e4f] 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 7: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.797 INFO:tasks.ceph.osd.2.smithi050.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T21:19:09.783+0000 7fde42359640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fde42359640 time 2024-04-02T21:19:09.782414+0000 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555c62440d8e] 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.800 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-04-02T21:19:09.799+0000 7fde42359640 -1 *** Caught signal (Aborted) ** 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: in thread 7fde42359640 thread_name:bstore_aio 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fde4f254db0] 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fde4f2a154c] 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: raise() 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: abort() 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555c62440e4f] 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.802 INFO:tasks.ceph.osd.2.smithi050.stderr: 7: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.803 INFO:tasks.ceph.osd.2.smithi050.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.803 INFO:tasks.ceph.osd.2.smithi050.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.803 INFO:tasks.ceph.osd.2.smithi050.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-02T21:19:09.803 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.833 INFO:tasks.ceph.osd.2.smithi050.stderr: -1664> 2024-04-02T21:19:09.782+0000 7fde42359640 -1 bdev(0x555c64650e00 /var/lib/ceph/osd/ceph-2/block) _aio_thread got r=-28 ((28) No space left on device) 2024-04-02T21:19:09.833 INFO:tasks.ceph.osd.2.smithi050.stderr: -1550> 2024-04-02T21:19:09.783+0000 7fde42359640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fde42359640 time 2024-04-02T21:19:09.782414+0000 2024-04-02T21:19:09.833 INFO:tasks.ceph.osd.2.smithi050.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:19:09.833 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.833 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.834 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555c62440d8e] 2024-04-02T21:19:09.834 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.834 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.834 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.834 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.834 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: -549> 2024-04-02T21:19:09.799+0000 7fde42359640 -1 *** Caught signal (Aborted) ** 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: in thread 7fde42359640 thread_name:bstore_aio 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fde4f254db0] 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fde4f2a154c] 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: raise() 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: abort() 2024-04-02T21:19:09.835 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555c62440e4f] 2024-04-02T21:19:09.836 INFO:tasks.ceph.osd.2.smithi050.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.836 INFO:tasks.ceph.osd.2.smithi050.stderr: 7: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.836 INFO:tasks.ceph.osd.2.smithi050.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.836 INFO:tasks.ceph.osd.2.smithi050.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.836 INFO:tasks.ceph.osd.2.smithi050.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-02T21:19:09.836 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.853 INFO:tasks.ceph.osd.2.smithi050.stderr: -2665> 2024-04-02T21:19:09.782+0000 7fde42359640 -1 bdev(0x555c64650e00 /var/lib/ceph/osd/ceph-2/block) _aio_thread got r=-28 ((28) No space left on device) 2024-04-02T21:19:09.853 INFO:tasks.ceph.osd.2.smithi050.stderr: -2551> 2024-04-02T21:19:09.783+0000 7fde42359640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fde42359640 time 2024-04-02T21:19:09.782414+0000 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2609-g191d963a/rpm/el9/BUILD/ceph-19.0.0-2609-g191d963a/src/blk/kernel/KernelDevice.cc: 666: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x555c62440d8e] 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.854 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.855 INFO:tasks.ceph.osd.2.smithi050.stderr: -1550> 2024-04-02T21:19:09.799+0000 7fde42359640 -1 *** Caught signal (Aborted) ** 2024-04-02T21:19:09.855 INFO:tasks.ceph.osd.2.smithi050.stderr: in thread 7fde42359640 thread_name:bstore_aio 2024-04-02T21:19:09.855 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:09.855 INFO:tasks.ceph.osd.2.smithi050.stderr: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T21:19:09.855 INFO:tasks.ceph.osd.2.smithi050.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fde4f254db0] 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fde4f2a154c] 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 3: raise() 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 4: abort() 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x555c62440e4f] 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x555c62e1c8f0] 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 7: ceph-osd(+0xdd1a61) [0x555c62e1ca61] 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fde4f29f802] 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fde4f23f450] 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-02T21:19:09.856 INFO:tasks.ceph.osd.2.smithi050.stderr: 2024-04-02T21:19:10.071 INFO:tasks.ceph.osd.2.smithi050.stderr:daemon-helper: command failed with exit status 5 2024-04-02T21:19:13.753 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-02T21:19:13.753 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~159s 2024-04-02T21:19:20.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~7s 2024-04-02T21:19:20.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~165s 2024-04-02T21:19:26.779 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~13s 2024-04-02T21:19:26.779 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~172s 2024-04-02T21:19:33.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~20s 2024-04-02T21:19:33.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~178s 2024-04-02T21:19:39.803 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~26s 2024-04-02T21:19:39.803 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~185s 2024-04-02T21:19:46.317 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~33s 2024-04-02T21:19:46.317 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~191s 2024-04-02T21:19:52.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~39s 2024-04-02T21:19:52.830 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~198s 2024-04-02T21:19:59.342 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~46s 2024-04-02T21:19:59.342 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~204s 2024-04-02T21:20:05.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~52s 2024-04-02T21:20:05.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~211s 2024-04-02T21:20:12.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~59s 2024-04-02T21:20:12.367 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~217s 2024-04-02T21:20:18.880 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2024-04-02T21:20:18.880 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~224s 2024-04-02T21:20:25.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~72s 2024-04-02T21:20:25.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~230s 2024-04-02T21:20:31.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~78s 2024-04-02T21:20:31.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~237s 2024-04-02T21:20:38.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~85s 2024-04-02T21:20:38.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~243s 2024-04-02T21:20:44.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~91s 2024-04-02T21:20:44.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~250s 2024-04-02T21:20:51.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~98s 2024-04-02T21:20:51.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~256s 2024-04-02T21:20:57.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~104s 2024-04-02T21:20:57.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~263s 2024-04-02T21:21:04.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~111s 2024-04-02T21:21:04.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~269s 2024-04-02T21:21:10.984 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~117s 2024-04-02T21:21:10.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~276s 2024-04-02T21:21:17.497 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~124s 2024-04-02T21:21:17.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~282s 2024-04-02T21:21:24.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~130s 2024-04-02T21:21:24.011 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~289s 2024-04-02T21:21:30.523 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~137s 2024-04-02T21:21:30.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~295s 2024-04-02T21:21:37.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~143s 2024-04-02T21:21:37.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~302s 2024-04-02T21:21:37.037 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-02T21:21:37.037 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T21:21:37.037 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T21:21:37.170 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T21:21:37.170 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T21:21:37.186 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T21:21:39.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.856+0000 7f8b7adff640 1 Processor -- start 2024-04-02T21:21:39.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.856+0000 7f8b7adff640 1 -- start start 2024-04-02T21:21:39.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b7adff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8b7415e370 0x7f8b740d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b7adff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7406d800 0x7f8b7409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b7adff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b7406c760 con 0x7f8b740d12f0 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b7adff640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7406d370 con 0x7f8b7415e370 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b7adff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7406ca40 con 0x7f8b7406d800 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b78b9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8b7415e370 0x7f8b740d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b78b9b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f8b7415e370 0x7f8b740d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50534/0 (socket says 172.21.15.50:50534) 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3784284064 learned_addr learned my addr 172.21.15.50:0/3784284064 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.857+0000 7f8b73fff640 1 --2- 172.21.15.50:0/3784284064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7406d800 0x7f8b7409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.858+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3784284064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7406d800 msgr2=0x7f8b7409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.858+0000 7f8b78b9b640 1 --2- 172.21.15.50:0/3784284064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7406d800 0x7f8b7409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.858+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3784284064 >> v1:172.21.15.188:6789/0 conn(0x7f8b740d12f0 legacy=0x7f8b74162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.858+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3784284064 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7406cd20 con 0x7f8b7415e370 2024-04-02T21:21:39.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.858+0000 7f8b78b9b640 1 --2- 172.21.15.50:0/3784284064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7415e370 0x7f8b740d0db0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009870 tx=0x7f8b6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf003f38f6a2197 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:39.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b737fe640 1 -- 172.21.15.50:0/3784284064 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64034070 con 0x7f8b7415e370 2024-04-02T21:21:39.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b737fe640 1 -- 172.21.15.50:0/3784284064 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b64030840 con 0x7f8b7415e370 2024-04-02T21:21:39.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b737fe640 1 -- 172.21.15.50:0/3784284064 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64030c10 con 0x7f8b7415e370 2024-04-02T21:21:39.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 -- 172.21.15.50:0/3784284064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7415e370 msgr2=0x7f8b740d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:39.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3784284064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7415e370 0x7f8b740d0db0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009870 tx=0x7f8b6402b980 comp rx=0 tx=0).stop 2024-04-02T21:21:39.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 -- 172.21.15.50:0/3784284064 shutdown_connections 2024-04-02T21:21:39.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3784284064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7406d800 0x7f8b7409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:39.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3784284064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7415e370 0x7f8b740d0db0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:39.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 -- 172.21.15.50:0/3784284064 >> 172.21.15.50:0/3784284064 conn(0x7f8b740b3ba0 msgr2=0x7f8b740b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:39.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 -- 172.21.15.50:0/3784284064 shutdown_connections 2024-04-02T21:21:39.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.859+0000 7f8b7adff640 1 -- 172.21.15.50:0/3784284064 wait complete. 2024-04-02T21:21:39.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.866+0000 7f8b7adff640 1 Processor -- start 2024-04-02T21:21:39.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.866+0000 7f8b7adff640 1 -- start start 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b7adff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7406d800 0x7f8b74153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b7adff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b740d12f0 0x7f8b740c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b7adff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7415e370 0x7f8b740b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b7adff640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7415e8d0 con 0x7f8b7406d800 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b7adff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7415e750 con 0x7f8b7415e370 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b7adff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7409af80 con 0x7f8b740d12f0 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7406d800 0x7f8b74153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:39.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7406d800 0x7f8b74153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50544/0 (socket says 172.21.15.50:50544) 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3030973815 learned_addr learned my addr 172.21.15.50:0/3030973815 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b73fff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b740d12f0 0x7f8b740c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b7939c640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7415e370 0x7f8b740b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b740d12f0 msgr2=0x7f8b740c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b740d12f0 0x7f8b740c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7415e370 msgr2=0x7f8b740b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7415e370 0x7f8b740b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.868+0000 7f8b78b9b640 1 -- 172.21.15.50:0/3030973815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b740c8ea0 con 0x7f8b7406d800 2024-04-02T21:21:39.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.869+0000 7f8b78b9b640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7406d800 0x7f8b74153230 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f8b6402b9c0 tx=0x7f8b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:39.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.878+0000 7f8b717fa640 1 -- 172.21.15.50:0/3030973815 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64034040 con 0x7f8b7406d800 2024-04-02T21:21:39.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.878+0000 7f8b717fa640 1 -- 172.21.15.50:0/3030973815 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b64039d90 con 0x7f8b7406d800 2024-04-02T21:21:39.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.878+0000 7f8b717fa640 1 -- 172.21.15.50:0/3030973815 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64043630 con 0x7f8b7406d800 2024-04-02T21:21:39.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.878+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b740c88e0 con 0x7f8b7406d800 2024-04-02T21:21:39.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.880+0000 7f8b717fa640 1 -- 172.21.15.50:0/3030973815 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b64037070 con 0x7f8b7406d800 2024-04-02T21:21:39.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.881+0000 7f8b717fa640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b6006c940 0x7f8b6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:39.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.882+0000 7f8b73fff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b6006c940 0x7f8b6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:39.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.882+0000 7f8b73fff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b6006c940 0x7f8b6006ee00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8b740b0cc0 tx=0x7f8b68009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.885+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7415fc90 con 0x7f8b7406d800 2024-04-02T21:21:39.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.885+0000 7f8b717fa640 1 -- 172.21.15.50:0/3030973815 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6130+0+0 (secure 0 0 0) 0x7f8b64079b00 con 0x7f8b7406d800 2024-04-02T21:21:39.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.888+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b740756f0 con 0x7f8b7406d800 2024-04-02T21:21:39.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:39.893+0000 7f8b717fa640 1 -- 172.21.15.50:0/3030973815 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8b6403e070 con 0x7f8b7406d800 2024-04-02T21:21:40.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.104+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b740b8030 con 0x7f8b7406d800 2024-04-02T21:21:40.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.105+0000 7f8b717fa640 1 -- 172.21.15.50:0/3030973815 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16792 (secure 0 0 0) 0x7f8b6407ba60 con 0x7f8b7406d800 2024-04-02T21:21:40.105 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T21:21:40.105 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":45,"fsid":"057e6c06-7c98-48a0-9312-9e11e9abe7fe","created":"2024-04-02T16:21:36.443519+0000","modified":"2024-04-02T21:19:11.502843+0000","last_up_change":"2024-04-02T21:19:10.495187+0000","last_in_change":"2024-04-02T16:21:38.692635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:21:48.681339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:23:13.826831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"error_message":"pool 5 has some PGs where all OSDs are with primary_affinity 0 (at least pg 5.6), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-02T16:23:14.911147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"error_message":"pool 6 has some PGs where all OSDs are with primary_affinity 0 (at least pg 6.31), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"05cd5072-d309-4ce6-9b2b-dbbd6dab447e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6810","nonce":3677185598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6814","nonce":3677185598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6822","nonce":3677185598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3677185598},{"type":"v1","addr":"172.21.15.50:6818","nonce":3677185598}]},"public_addr":"172.21.15.50:6810/3677185598","cluster_addr":"172.21.15.50:6814/3677185598","heartbeat_back_addr":"172.21.15.50:6822/3677185598","heartbeat_front_addr":"172.21.15.50:6818/3677185598","state":["exists","up"]},{"osd":1,"uuid":"dd26535d-1ca6-4547-974f-43bec7782174","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6809","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6811","nonce":3280203769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6813","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6815","nonce":3280203769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6821","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6823","nonce":3280203769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6817","nonce":3280203769},{"type":"v1","addr":"172.21.15.50:6819","nonce":3280203769}]},"public_addr":"172.21.15.50:6811/3280203769","cluster_addr":"172.21.15.50:6815/3280203769","heartbeat_back_addr":"172.21.15.50:6823/3280203769","heartbeat_front_addr":"172.21.15.50:6819/3280203769","state":["exists","up"]},{"osd":2,"uuid":"54c49032-e743-4090-8c7b-a09f6bd14a69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6825","nonce":815998529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6827","nonce":815998529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6831","nonce":815998529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":815998529},{"type":"v1","addr":"172.21.15.50:6829","nonce":815998529}]},"public_addr":"172.21.15.50:6825/815998529","cluster_addr":"172.21.15.50:6827/815998529","heartbeat_back_addr":"172.21.15.50:6831/815998529","heartbeat_front_addr":"172.21.15.50:6829/815998529","state":["exists"]},{"osd":3,"uuid":"ad773d48-a65b-4b99-8d48-8b8bdf1f4853","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6801","nonce":2132969240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6803","nonce":2132969240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6807","nonce":2132969240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2132969240},{"type":"v1","addr":"172.21.15.50:6805","nonce":2132969240}]},"public_addr":"172.21.15.50:6801/2132969240","cluster_addr":"172.21.15.50:6803/2132969240","heartbeat_back_addr":"172.21.15.50:6807/2132969240","heartbeat_front_addr":"172.21.15.50:6805/2132969240","state":["exists","up"]},{"osd":4,"uuid":"7d37ca27-5a6f-4c8e-a029-3076f76e0166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6825","nonce":1410600395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6827","nonce":1410600395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6831","nonce":1410600395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":1410600395},{"type":"v1","addr":"172.21.15.188:6829","nonce":1410600395}]},"public_addr":"172.21.15.188:6825/1410600395","cluster_addr":"172.21.15.188:6827/1410600395","heartbeat_back_addr":"172.21.15.188:6831/1410600395","heartbeat_front_addr":"172.21.15.188:6829/1410600395","state":["exists","up"]},{"osd":5,"uuid":"f4e15f5c-b81a-46b9-884d-b0fc57937947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":42,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6817","nonce":924583379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6819","nonce":924583379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6823","nonce":924583379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":924583379},{"type":"v1","addr":"172.21.15.188:6821","nonce":924583379}]},"public_addr":"172.21.15.188:6817/924583379","cluster_addr":"172.21.15.188:6819/924583379","heartbeat_back_addr":"172.21.15.188:6823/924583379","heartbeat_front_addr":"172.21.15.188:6821/924583379","state":["exists"]},{"osd":6,"uuid":"427efa9c-ad3f-4f2b-8205-f1640d4e55ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6809","nonce":3730321182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6811","nonce":3730321182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6815","nonce":3730321182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3730321182},{"type":"v1","addr":"172.21.15.188:6813","nonce":3730321182}]},"public_addr":"172.21.15.188:6809/3730321182","cluster_addr":"172.21.15.188:6811/3730321182","heartbeat_back_addr":"172.21.15.188:6815/3730321182","heartbeat_front_addr":"172.21.15.188:6813/3730321182","state":["exists","up"]},{"osd":7,"uuid":"5080b25c-8467-43c2-b078-f006968201f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6801","nonce":2935292990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6803","nonce":2935292990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6807","nonce":2935292990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2935292990},{"type":"v1","addr":"172.21.15.188:6805","nonce":2935292990}]},"public_addr":"172.21.15.188:6801/2935292990","cluster_addr":"172.21.15.188:6803/2935292990","heartbeat_back_addr":"172.21.15.188:6807/2935292990","heartbeat_front_addr":"172.21.15.188:6805/2935292990","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.917558+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.861114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.866831+0000","dead_epoch":44},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.858563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.872865+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.855809+0000","dead_epoch":42},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.802926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:21:43.848588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b6006c940 msgr2=0x7f8b6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b6006c940 0x7f8b6006ee00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8b740b0cc0 tx=0x7f8b68009190 comp rx=0 tx=0).stop 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7406d800 msgr2=0x7f8b74153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7406d800 0x7f8b74153230 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f8b6402b9c0 tx=0x7f8b640046c0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 shutdown_connections 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7415e370 0x7f8b740b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b740d12f0 0x7f8b740c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f8b6006c940 0x7f8b6006ee00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 --2- 172.21.15.50:0/3030973815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8b7406d800 0x7f8b74153230 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 >> 172.21.15.50:0/3030973815 conn(0x7f8b740b3ba0 msgr2=0x7f8b740aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:40.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 shutdown_connections 2024-04-02T21:21:40.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.107+0000 7f8b7adff640 1 -- 172.21.15.50:0/3030973815 wait complete. 2024-04-02T21:21:40.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:21:48.681339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:23:13.826831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'error_message': 'pool 5 has some PGs where all OSDs are with primary_affinity 0 (at least pg 5.6), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:23:14.911147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'error_message': 'pool 6 has some PGs where all OSDs are with primary_affinity 0 (at least pg 6.31), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}] 2024-04-02T21:21:40.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T21:21:40.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.252+0000 7f6174e38640 1 Processor -- start 2024-04-02T21:21:40.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.252+0000 7f6174e38640 1 -- start start 2024-04-02T21:21:40.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.252+0000 7f6174e38640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f617015d320 0x7f617006f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:40.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.252+0000 7f6174e38640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61700b2cb0 0x7f61700b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:40.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.252+0000 7f6174e38640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6170160520 con 0x7f6170145be0 2024-04-02T21:21:40.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.252+0000 7f6174e38640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f61700b3640 con 0x7f617015d320 2024-04-02T21:21:40.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.252+0000 7f6174e38640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f61700cfd20 con 0x7f61700b2cb0 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f6167fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f617015d320 0x7f617006f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f6167fff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f617015d320 0x7f617006f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50558/0 (socket says 172.21.15.50:50558) 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f6167fff640 1 -- 172.21.15.50:0/858549542 learned_addr learned my addr 172.21.15.50:0/858549542 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f616ed9d640 1 --2- 172.21.15.50:0/858549542 >> v2:172.21.15.188:3300/0 conn(0x7f61700b2cb0 0x7f61700b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f6167fff640 1 -- 172.21.15.50:0/858549542 >> v2:172.21.15.188:3300/0 conn(0x7f61700b2cb0 msgr2=0x7f61700b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f6167fff640 1 --2- 172.21.15.50:0/858549542 >> v2:172.21.15.188:3300/0 conn(0x7f61700b2cb0 0x7f61700b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f6167fff640 1 -- 172.21.15.50:0/858549542 >> v1:172.21.15.188:6789/0 conn(0x7f6170145be0 legacy=0x7f617015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f6167fff640 1 -- 172.21.15.50:0/858549542 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6170160800 con 0x7f617015d320 2024-04-02T21:21:40.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.253+0000 7f616ed9d640 1 --2- 172.21.15.50:0/858549542 >> v2:172.21.15.188:3300/0 conn(0x7f61700b2cb0 0x7f61700b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T21:21:40.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f6167fff640 1 --2- 172.21.15.50:0/858549542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f617006f920 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f61700d0f00 tx=0x7f615802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b2c17c40b0b373 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:40.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f616dd9b640 1 -- 172.21.15.50:0/858549542 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158035020 con 0x7f617015d320 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f616dd9b640 1 -- 172.21.15.50:0/858549542 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6158031870 con 0x7f617015d320 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f616dd9b640 1 -- 172.21.15.50:0/858549542 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158031c40 con 0x7f617015d320 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f6174e38640 1 -- 172.21.15.50:0/858549542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 msgr2=0x7f617006f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f6174e38640 1 --2- 172.21.15.50:0/858549542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f617006f920 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f61700d0f00 tx=0x7f615802b910 comp rx=0 tx=0).stop 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f6174e38640 1 -- 172.21.15.50:0/858549542 shutdown_connections 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f6174e38640 1 --2- 172.21.15.50:0/858549542 >> v2:172.21.15.188:3300/0 conn(0x7f61700b2cb0 0x7f61700b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f6174e38640 1 --2- 172.21.15.50:0/858549542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f617006f920 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.254+0000 7f6174e38640 1 -- 172.21.15.50:0/858549542 >> 172.21.15.50:0/858549542 conn(0x7f617006e6c0 msgr2=0x7f617006eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.255+0000 7f6174e38640 1 -- 172.21.15.50:0/858549542 shutdown_connections 2024-04-02T21:21:40.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.255+0000 7f6174e38640 1 -- 172.21.15.50:0/858549542 wait complete. 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.256+0000 7f6174e38640 1 Processor -- start 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.256+0000 7f6174e38640 1 -- start start 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6174e38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61700b2cb0 0x7f61701332e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6174e38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6170145be0 0x7f617015c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6174e38640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f61700ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6174e38640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f617006fe60 con 0x7f617015d320 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6174e38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61700cf940 con 0x7f6170145be0 2024-04-02T21:21:40.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6174e38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f61700d01b0 con 0x7f61700b2cb0 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f616ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f61700ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f616ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f61700ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50572/0 (socket says 172.21.15.50:50572) 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f616ed9d640 1 -- 172.21.15.50:0/1885314587 learned_addr learned my addr 172.21.15.50:0/1885314587 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6167fff640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6170145be0 0x7f617015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f616e59c640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61700b2cb0 0x7f61701332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6167fff640 1 -- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61700b2cb0 msgr2=0x7f61701332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6167fff640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61700b2cb0 0x7f61701332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6167fff640 1 -- 172.21.15.50:0/1885314587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 msgr2=0x7f61700ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6167fff640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f61700ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f6167fff640 1 -- 172.21.15.50:0/1885314587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61700abe70 con 0x7f6170145be0 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.257+0000 7f616ed9d640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f61700ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.258+0000 7f6167fff640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6170145be0 0x7f617015c910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6158033000 tx=0x7f61580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.258+0000 7f6166ffd640 1 -- 172.21.15.50:0/1885314587 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158040070 con 0x7f6170145be0 2024-04-02T21:21:40.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.258+0000 7f6166ffd640 1 -- 172.21.15.50:0/1885314587 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615803a420 con 0x7f6170145be0 2024-04-02T21:21:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.258+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61700ac150 con 0x7f6170145be0 2024-04-02T21:21:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.258+0000 7f6166ffd640 1 -- 172.21.15.50:0/1885314587 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158035040 con 0x7f6170145be0 2024-04-02T21:21:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.258+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61700ac710 con 0x7f6170145be0 2024-04-02T21:21:40.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.259+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6170075760 con 0x7f6170145be0 2024-04-02T21:21:40.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.259+0000 7f6166ffd640 1 -- 172.21.15.50:0/1885314587 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f615804b060 con 0x7f6170145be0 2024-04-02T21:21:40.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.260+0000 7f6166ffd640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f615006c8f0 0x7f615006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:40.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.260+0000 7f6166ffd640 1 -- 172.21.15.50:0/1885314587 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6130+0+0 (secure 0 0 0) 0x7f61580adae0 con 0x7f6170145be0 2024-04-02T21:21:40.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.260+0000 7f616e59c640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f615006c8f0 0x7f615006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:40.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.261+0000 7f616e59c640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f615006c8f0 0x7f615006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f61600017f0 tx=0x7f6160009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:40.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.265+0000 7f6166ffd640 1 -- 172.21.15.50:0/1885314587 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f615807b710 con 0x7f6170145be0 2024-04-02T21:21:40.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:40.671+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6170072730 con 0x7f6170145be0 2024-04-02T21:21:41.069 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-04-02T21:21:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.067+0000 7f6166ffd640 1 -- 172.21.15.50:0/1885314587 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f615807fa10 con 0x7f6170145be0 2024-04-02T21:21:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f615006c8f0 msgr2=0x7f615006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f615006c8f0 0x7f615006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f61600017f0 tx=0x7f6160009040 comp rx=0 tx=0).stop 2024-04-02T21:21:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6170145be0 msgr2=0x7f617015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6170145be0 0x7f617015c910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6158033000 tx=0x7f61580046c0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 shutdown_connections 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f615006c8f0 0x7f615006edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f617015d320 0x7f61700ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6170145be0 0x7f617015c910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 --2- 172.21.15.50:0/1885314587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61700b2cb0 0x7f61701332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.070+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 >> 172.21.15.50:0/1885314587 conn(0x7f617006e6c0 msgr2=0x7f61700b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.071+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 shutdown_connections 2024-04-02T21:21:41.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.071+0000 7f6174e38640 1 -- 172.21.15.50:0/1885314587 wait complete. 2024-04-02T21:21:41.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T21:21:41.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.337+0000 7f9d42688640 1 Processor -- start 2024-04-02T21:21:41.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.337+0000 7f9d42688640 1 -- start start 2024-04-02T21:21:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.338+0000 7f9d42688640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9d3c145be0 0x7f9d3c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.338+0000 7f9d42688640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d3c15fcf0 0x7f9d3c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.338+0000 7f9d42688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c0b2cb0 0x7f9d3c0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.338+0000 7f9d42688640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f9d3c0cfd20 con 0x7f9d3c145be0 2024-04-02T21:21:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.338+0000 7f9d42688640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9d3c160520 con 0x7f9d3c15fcf0 2024-04-02T21:21:41.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.338+0000 7f9d42688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c160800 con 0x7f9d3c0b2cb0 2024-04-02T21:21:41.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.339+0000 7f9d3bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9d3c145be0 0x7f9d3c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.339+0000 7f9d3bfff640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f9d3c145be0 0x7f9d3c15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50576/0 (socket says 172.21.15.50:50576) 2024-04-02T21:21:41.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.339+0000 7f9d3bfff640 1 -- 172.21.15.50:0/4107912715 learned_addr learned my addr 172.21.15.50:0/4107912715 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:41.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.339+0000 7f9d40c25640 1 --2- 172.21.15.50:0/4107912715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c0b2cb0 0x7f9d3c0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.339+0000 7f9d3b7fe640 1 --2- 172.21.15.50:0/4107912715 >> v2:172.21.15.188:3300/0 conn(0x7f9d3c15fcf0 0x7f9d3c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.340+0000 7f9d3bfff640 1 -- 172.21.15.50:0/4107912715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c0b2cb0 msgr2=0x7f9d3c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T21:21:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.340+0000 7f9d3bfff640 1 --2- 172.21.15.50:0/4107912715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c0b2cb0 0x7f9d3c0b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.340+0000 7f9d3bfff640 1 -- 172.21.15.50:0/4107912715 >> v2:172.21.15.188:3300/0 conn(0x7f9d3c15fcf0 msgr2=0x7f9d3c06f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T21:21:41.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.340+0000 7f9d3bfff640 1 --2- 172.21.15.50:0/4107912715 >> v2:172.21.15.188:3300/0 conn(0x7f9d3c15fcf0 0x7f9d3c06f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.340+0000 7f9d3bfff640 1 -- 172.21.15.50:0/4107912715 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c160ae0 con 0x7f9d3c145be0 2024-04-02T21:21:41.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.342+0000 7f9d3bfff640 1 --2- 172.21.15.50:0/4107912715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c15d320 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c06f610 tx=0x7f9d3002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cd5620007209c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:41.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.342+0000 7f9d3affd640 1 -- 172.21.15.50:0/4107912715 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30035020 con 0x7f9d3c145be0 2024-04-02T21:21:41.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.343+0000 7f9d3affd640 1 -- 172.21.15.50:0/4107912715 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d30031880 con 0x7f9d3c145be0 2024-04-02T21:21:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.344+0000 7f9d42688640 1 -- 172.21.15.50:0/4107912715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 msgr2=0x7f9d3c15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.344+0000 7f9d3affd640 1 -- 172.21.15.50:0/4107912715 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30035020 con 0x7f9d3c145be0 2024-04-02T21:21:41.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.344+0000 7f9d42688640 1 --2- 172.21.15.50:0/4107912715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c15d320 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c06f610 tx=0x7f9d3002b920 comp rx=0 tx=0).stop 2024-04-02T21:21:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.347+0000 7f9d42688640 1 -- 172.21.15.50:0/4107912715 shutdown_connections 2024-04-02T21:21:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.347+0000 7f9d42688640 1 --2- 172.21.15.50:0/4107912715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c0b2cb0 0x7f9d3c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.347+0000 7f9d42688640 1 --2- 172.21.15.50:0/4107912715 >> v2:172.21.15.188:3300/0 conn(0x7f9d3c15fcf0 0x7f9d3c06f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.347+0000 7f9d42688640 1 --2- 172.21.15.50:0/4107912715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c15d320 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.347+0000 7f9d42688640 1 -- 172.21.15.50:0/4107912715 >> 172.21.15.50:0/4107912715 conn(0x7f9d3c06e6c0 msgr2=0x7f9d3c06eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:41.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.349+0000 7f9d42688640 1 -- 172.21.15.50:0/4107912715 shutdown_connections 2024-04-02T21:21:41.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.351+0000 7f9d42688640 1 -- 172.21.15.50:0/4107912715 wait complete. 2024-04-02T21:21:41.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.353+0000 7f9d42688640 1 Processor -- start 2024-04-02T21:21:41.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.356+0000 7f9d42688640 1 -- start start 2024-04-02T21:21:41.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.357+0000 7f9d42688640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3c0b2cb0 0x7f9d3c1f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.357+0000 7f9d42688640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c1f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.357+0000 7f9d42688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c15fcf0 0x7f9d3c1f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.357+0000 7f9d42688640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c160180 con 0x7f9d3c145be0 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.357+0000 7f9d42688640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c06fe60 con 0x7f9d3c0b2cb0 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.357+0000 7f9d42688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c160300 con 0x7f9d3c15fcf0 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.358+0000 7f9d3b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c1f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.358+0000 7f9d3b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c1f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50586/0 (socket says 172.21.15.50:50586) 2024-04-02T21:21:41.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.358+0000 7f9d3b7fe640 1 -- 172.21.15.50:0/1120570128 learned_addr learned my addr 172.21.15.50:0/1120570128 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.359+0000 7f9d3b7fe640 1 -- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c15fcf0 msgr2=0x7f9d3c1f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.359+0000 7f9d3b7fe640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c15fcf0 0x7f9d3c1f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.359+0000 7f9d3b7fe640 1 -- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3c0b2cb0 msgr2=0x7f9d3c1f3c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T21:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.359+0000 7f9d3b7fe640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3c0b2cb0 0x7f9d3c1f3c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.359+0000 7f9d3b7fe640 1 -- 172.21.15.50:0/1120570128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c1f6090 con 0x7f9d3c145be0 2024-04-02T21:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.359+0000 7f9d3b7fe640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c1f54b0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c007e40 tx=0x7f9d2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:41.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.360+0000 7f9d38ff9640 1 -- 172.21.15.50:0/1120570128 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c014620 con 0x7f9d3c145be0 2024-04-02T21:21:41.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.360+0000 7f9d38ff9640 1 -- 172.21.15.50:0/1120570128 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2c019070 con 0x7f9d3c145be0 2024-04-02T21:21:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.361+0000 7f9d38ff9640 1 -- 172.21.15.50:0/1120570128 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0149e0 con 0x7f9d3c145be0 2024-04-02T21:21:41.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.361+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3c1f6370 con 0x7f9d3c145be0 2024-04-02T21:21:41.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.361+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3c1f6930 con 0x7f9d3c145be0 2024-04-02T21:21:41.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.365+0000 7f9d38ff9640 1 -- 172.21.15.50:0/1120570128 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d2c014b80 con 0x7f9d3c145be0 2024-04-02T21:21:41.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.365+0000 7f9d38ff9640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d2806c9c0 0x7f9d2806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.366+0000 7f9d38ff9640 1 -- 172.21.15.50:0/1120570128 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6130+0+0 (secure 0 0 0) 0x7f9d2c090170 con 0x7f9d3c145be0 2024-04-02T21:21:41.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.369+0000 7f9d3bfff640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d2806c9c0 0x7f9d2806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.370+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3c075760 con 0x7f9d3c145be0 2024-04-02T21:21:41.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.377+0000 7f9d3bfff640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d2806c9c0 0x7f9d2806ee80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9d30036000 tx=0x7f9d30008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:41.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.380+0000 7f9d38ff9640 1 -- 172.21.15.50:0/1120570128 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9d2c05a010 con 0x7f9d3c145be0 2024-04-02T21:21:41.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.669+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d3c0ce380 con 0x7f9d3c145be0 2024-04-02T21:21:41.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.670+0000 7f9d38ff9640 1 -- 172.21.15.50:0/1120570128 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) ==== 104+0+11 (secure 0 0 0) 0x7f9d2c059910 con 0x7f9d3c145be0 2024-04-02T21:21:41.669 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-04-02T21:21:41.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d2806c9c0 msgr2=0x7f9d2806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d2806c9c0 0x7f9d2806ee80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9d30036000 tx=0x7f9d30008030 comp rx=0 tx=0).stop 2024-04-02T21:21:41.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 msgr2=0x7f9d3c1f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c1f54b0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c007e40 tx=0x7f9d2c00e4b0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 shutdown_connections 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d3c15fcf0 0x7f9d3c1f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f9d2806c9c0 0x7f9d2806ee80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d3c145be0 0x7f9d3c1f54b0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 --2- 172.21.15.50:0/1120570128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3c0b2cb0 0x7f9d3c1f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 >> 172.21.15.50:0/1120570128 conn(0x7f9d3c06e6c0 msgr2=0x7f9d3c0b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 shutdown_connections 2024-04-02T21:21:41.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.672+0000 7f9d42688640 1 -- 172.21.15.50:0/1120570128 wait complete. 2024-04-02T21:21:41.686 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T21:21:41.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.807+0000 7f1b8d27b640 1 Processor -- start 2024-04-02T21:21:41.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.807+0000 7f1b8d27b640 1 -- start start 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b8d27b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b880d0cc0 0x7f1b880d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b8d27b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b880b1e60 0x7f1b880b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b8d27b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b880b10e0 con 0x7f1b88161c80 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b8d27b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f1b880d1660 con 0x7f1b880d0cc0 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b8d27b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1b880b1380 con 0x7f1b880b1e60 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b867fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b880d0cc0 0x7f1b880d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b867fc640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f1b880d0cc0 0x7f1b880d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50588/0 (socket says 172.21.15.50:50588) 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b867fc640 1 -- 172.21.15.50:0/3263008310 learned_addr learned my addr 172.21.15.50:0/3263008310 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b877fe640 1 --2- 172.21.15.50:0/3263008310 >> v2:172.21.15.188:3300/0 conn(0x7f1b880b1e60 0x7f1b880b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b867fc640 1 -- 172.21.15.50:0/3263008310 >> v2:172.21.15.188:3300/0 conn(0x7f1b880b1e60 msgr2=0x7f1b880b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T21:21:41.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b867fc640 1 --2- 172.21.15.50:0/3263008310 >> v2:172.21.15.188:3300/0 conn(0x7f1b880b1e60 0x7f1b880b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b867fc640 1 -- 172.21.15.50:0/3263008310 >> v1:172.21.15.188:6789/0 conn(0x7f1b88161c80 legacy=0x7f1b880d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b867fc640 1 -- 172.21.15.50:0/3263008310 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b880b1660 con 0x7f1b880d0cc0 2024-04-02T21:21:41.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.808+0000 7f1b877fe640 1 --2- 172.21.15.50:0/3263008310 >> v2:172.21.15.188:3300/0 conn(0x7f1b880b1e60 0x7f1b880b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.809+0000 7f1b867fc640 1 --2- 172.21.15.50:0/3263008310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880d1120 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f1b8808e770 tx=0x7f1b7802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d874fa346df164c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.809+0000 7f1b85ffb640 1 -- 172.21.15.50:0/3263008310 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78035020 con 0x7f1b880d0cc0 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.809+0000 7f1b85ffb640 1 -- 172.21.15.50:0/3263008310 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b78031870 con 0x7f1b880d0cc0 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.809+0000 7f1b85ffb640 1 -- 172.21.15.50:0/3263008310 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78031c40 con 0x7f1b880d0cc0 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 -- 172.21.15.50:0/3263008310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 msgr2=0x7f1b880d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/3263008310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880d1120 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f1b8808e770 tx=0x7f1b7802b910 comp rx=0 tx=0).stop 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 -- 172.21.15.50:0/3263008310 shutdown_connections 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/3263008310 >> v2:172.21.15.188:3300/0 conn(0x7f1b880b1e60 0x7f1b880b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/3263008310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880d1120 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 -- 172.21.15.50:0/3263008310 >> 172.21.15.50:0/3263008310 conn(0x7f1b8809d5e0 msgr2=0x7f1b880d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 -- 172.21.15.50:0/3263008310 shutdown_connections 2024-04-02T21:21:41.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.810+0000 7f1b8d27b640 1 -- 172.21.15.50:0/3263008310 wait complete. 2024-04-02T21:21:41.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.811+0000 7f1b8d27b640 1 Processor -- start 2024-04-02T21:21:41.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b8d27b640 1 -- start start 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b8d27b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b880b1e60 0x7f1b8815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b8d27b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b8d27b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b88161c80 0x7f1b880ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b8d27b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880d0180 con 0x7f1b880d0cc0 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b8d27b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880d3340 con 0x7f1b88161c80 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b8d27b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b880d0300 con 0x7f1b880b1e60 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b867fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50598/0 (socket says 172.21.15.50:50598) 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.812+0000 7f1b867fc640 1 -- 172.21.15.50:0/672024321 learned_addr learned my addr 172.21.15.50:0/672024321 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b867fc640 1 -- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b880b1e60 msgr2=0x7f1b8815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b867fc640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b880b1e60 0x7f1b8815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b867fc640 1 -- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b88161c80 msgr2=0x7f1b880ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b867fc640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b88161c80 0x7f1b880ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b867fc640 1 -- 172.21.15.50:0/672024321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8806ccd0 con 0x7f1b880d0cc0 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b867fc640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880ca340 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000c00 tx=0x7f1b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:41.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b7f7fe640 1 -- 172.21.15.50:0/672024321 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7803ad60 con 0x7f1b880d0cc0 2024-04-02T21:21:41.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b7f7fe640 1 -- 172.21.15.50:0/672024321 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b780443f0 con 0x7f1b880d0cc0 2024-04-02T21:21:41.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b880c83b0 con 0x7f1b880d0cc0 2024-04-02T21:21:41.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b7f7fe640 1 -- 172.21.15.50:0/672024321 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780447e0 con 0x7f1b880d0cc0 2024-04-02T21:21:41.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.813+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8806d250 con 0x7f1b880d0cc0 2024-04-02T21:21:41.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.814+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b88075760 con 0x7f1b880d0cc0 2024-04-02T21:21:41.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.815+0000 7f1b7f7fe640 1 -- 172.21.15.50:0/672024321 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b7804b060 con 0x7f1b880d0cc0 2024-04-02T21:21:41.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.815+0000 7f1b7f7fe640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b6806c8f0 0x7f1b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:41.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.815+0000 7f1b7f7fe640 1 -- 172.21.15.50:0/672024321 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6130+0+0 (secure 0 0 0) 0x7f1b780ade90 con 0x7f1b880d0cc0 2024-04-02T21:21:41.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.815+0000 7f1b86ffd640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b6806c8f0 0x7f1b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:41.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.816+0000 7f1b86ffd640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b6806c8f0 0x7f1b6806edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1b70000f60 tx=0x7f1b700079d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:41.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:41.820+0000 7f1b7f7fe640 1 -- 172.21.15.50:0/672024321 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1b7803f070 con 0x7f1b880d0cc0 2024-04-02T21:21:42.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.021+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1b880d39b0 con 0x7f1b880d0cc0 2024-04-02T21:21:42.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.022+0000 7f1b7f7fe640 1 -- 172.21.15.50:0/672024321 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) ==== 100+0+11 (secure 0 0 0) 0x7f1b7807c120 con 0x7f1b880d0cc0 2024-04-02T21:21:42.021 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-04-02T21:21:42.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b6806c8f0 msgr2=0x7f1b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b6806c8f0 0x7f1b6806edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1b70000f60 tx=0x7f1b700079d0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 msgr2=0x7f1b880ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880ca340 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000c00 tx=0x7f1b780046c0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 shutdown_connections 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b88161c80 0x7f1b880ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f1b6806c8f0 0x7f1b6806edb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b880d0cc0 0x7f1b880ca340 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 --2- 172.21.15.50:0/672024321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b880b1e60 0x7f1b8815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 >> 172.21.15.50:0/672024321 conn(0x7f1b8809d5e0 msgr2=0x7f1b880b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 shutdown_connections 2024-04-02T21:21:42.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.024+0000 7f1b8d27b640 1 -- 172.21.15.50:0/672024321 wait complete. 2024-04-02T21:21:42.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T21:21:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.153+0000 7f410941b640 1 Processor -- start 2024-04-02T21:21:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.153+0000 7f410941b640 1 -- start start 2024-04-02T21:21:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.153+0000 7f410941b640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f41040d1120 0x7f41040b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f410941b640 1 -- --> v1:172.21.15.50:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040b10e0 con 0x7f4104161c80 2024-04-02T21:21:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f410941b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040b1380 con 0x7f41040d0cc0 2024-04-02T21:21:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f410941b640 1 -- --> v2:172.21.15.50:3300/0 -- mon_getmap magic: 0 -- 0x7f41040b1660 con 0x7f41040d1120 2024-04-02T21:21:42.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f41037fe640 1 --2- >> v2:172.21.15.50:3300/0 conn(0x7f41040d1120 0x7f41040b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f4102ffd640 1 --1- >> v1:172.21.15.50:6789/0 conn(0x7f4104161c80 0x7f41040d2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.50:6789/0 says I am v1:172.21.15.50:50770/0 (socket says 172.21.15.50:50770) 2024-04-02T21:21:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f4102ffd640 1 -- 172.21.15.50:0/1959764583 learned_addr learned my addr 172.21.15.50:0/1959764583 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-04-02T21:21:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f41037fe640 1 -- 172.21.15.50:0/1959764583 >> v1:172.21.15.188:6789/0 conn(0x7f41040d0cc0 legacy=0x7f41040b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T21:21:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f41037fe640 1 -- 172.21.15.50:0/1959764583 >> v1:172.21.15.50:6789/0 conn(0x7f4104161c80 legacy=0x7f41040d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.154+0000 7f41037fe640 1 -- 172.21.15.50:0/1959764583 --> v2:172.21.15.50:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040d15b0 con 0x7f41040d1120 2024-04-02T21:21:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.155+0000 7f41037fe640 1 --2- 172.21.15.50:0/1959764583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040b2570 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009d90 tx=0x7f40fc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dfb41581d26c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.155+0000 7f4101ffb640 1 -- 172.21.15.50:0/1959764583 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc039070 con 0x7f41040d1120 2024-04-02T21:21:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.155+0000 7f4101ffb640 1 -- 172.21.15.50:0/1959764583 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc02cbd0 con 0x7f41040d1120 2024-04-02T21:21:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.155+0000 7f4101ffb640 1 -- 172.21.15.50:0/1959764583 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc034440 con 0x7f41040d1120 2024-04-02T21:21:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.156+0000 7f410941b640 1 -- 172.21.15.50:0/1959764583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 msgr2=0x7f41040b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.156+0000 7f410941b640 1 --2- 172.21.15.50:0/1959764583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040b2570 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009d90 tx=0x7f40fc02be20 comp rx=0 tx=0).stop 2024-04-02T21:21:42.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.156+0000 7f410941b640 1 -- 172.21.15.50:0/1959764583 shutdown_connections 2024-04-02T21:21:42.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.156+0000 7f410941b640 1 --2- 172.21.15.50:0/1959764583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040b2570 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.156+0000 7f410941b640 1 -- 172.21.15.50:0/1959764583 >> 172.21.15.50:0/1959764583 conn(0x7f410409d5e0 msgr2=0x7f41040d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:42.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.156+0000 7f410941b640 1 -- 172.21.15.50:0/1959764583 shutdown_connections 2024-04-02T21:21:42.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.156+0000 7f410941b640 1 -- 172.21.15.50:0/1959764583 wait complete. 2024-04-02T21:21:42.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.157+0000 7f410941b640 1 Processor -- start 2024-04-02T21:21:42.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.157+0000 7f410941b640 1 -- start start 2024-04-02T21:21:42.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f410941b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41040d0cc0 0x7f4104155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:42.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f410941b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:42.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f410941b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4104161c80 0x7f410406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:42.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f410941b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f41040d0180 con 0x7f41040d1120 2024-04-02T21:21:42.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f410941b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41040d3340 con 0x7f41040d0cc0 2024-04-02T21:21:42.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f410941b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41040d0300 con 0x7f4104161c80 2024-04-02T21:21:42.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50612/0 (socket says 172.21.15.50:50612) 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 -- 172.21.15.50:0/3549617154 learned_addr learned my addr 172.21.15.50:0/3549617154 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41037fe640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4104161c80 0x7f410406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f4102ffd640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41040d0cc0 0x7f4104155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 -- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4104161c80 msgr2=0x7f410406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4104161c80 0x7f410406c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 -- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41040d0cc0 msgr2=0x7f4104155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41040d0cc0 0x7f4104155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 -- 172.21.15.50:0/3549617154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040cbea0 con 0x7f41040d1120 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41037fe640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4104161c80 0x7f410406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f4102ffd640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41040d0cc0 0x7f4104155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f41027fc640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040cb450 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f40f800b3f0 tx=0x7f40f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.159+0000 7f40eb7fe640 1 -- 172.21.15.50:0/3549617154 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8015020 con 0x7f41040d1120 2024-04-02T21:21:42.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.160+0000 7f40eb7fe640 1 -- 172.21.15.50:0/3549617154 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f8011830 con 0x7f41040d1120 2024-04-02T21:21:42.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.160+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41040cc180 con 0x7f41040d1120 2024-04-02T21:21:42.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.160+0000 7f40eb7fe640 1 -- 172.21.15.50:0/3549617154 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011c20 con 0x7f41040d1120 2024-04-02T21:21:42.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.160+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41040cc740 con 0x7f41040d1120 2024-04-02T21:21:42.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.161+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4104075760 con 0x7f41040d1120 2024-04-02T21:21:42.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.162+0000 7f40eb7fe640 1 -- 172.21.15.50:0/3549617154 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40f8018020 con 0x7f41040d1120 2024-04-02T21:21:42.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.162+0000 7f40eb7fe640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40f006c8f0 0x7f40f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T21:21:42.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.162+0000 7f40eb7fe640 1 -- 172.21.15.50:0/3549617154 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6130+0+0 (secure 0 0 0) 0x7f40f808b910 con 0x7f41040d1120 2024-04-02T21:21:42.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.162+0000 7f4102ffd640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40f006c8f0 0x7f40f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T21:21:42.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.163+0000 7f4102ffd640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40f006c8f0 0x7f40f006edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f41040b7480 tx=0x7f40f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T21:21:42.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.169+0000 7f40eb7fe640 1 -- 172.21.15.50:0/3549617154 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f40f801f030 con 0x7f41040d1120 2024-04-02T21:21:42.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.396+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41040d4490 con 0x7f41040d1120 2024-04-02T21:21:42.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.396+0000 7f40eb7fe640 1 -- 172.21.15.50:0/3549617154 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f40f80172c0 con 0x7f41040d1120 2024-04-02T21:21:42.396 INFO:teuthology.orchestra.run.smithi050.stderr:listed 0 entries 2024-04-02T21:21:42.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.398+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40f006c8f0 msgr2=0x7f40f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.398+0000 7f410941b640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40f006c8f0 0x7f40f006edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f41040b7480 tx=0x7f40f4009040 comp rx=0 tx=0).stop 2024-04-02T21:21:42.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 msgr2=0x7f41040cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T21:21:42.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040cb450 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f40f800b3f0 tx=0x7f40f800b8b0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 shutdown_connections 2024-04-02T21:21:42.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4104161c80 0x7f410406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:6832/2785600878,v1:172.21.15.188:6833/2785600878] conn(0x7f40f006c8f0 0x7f40f006edb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41040d1120 0x7f41040cb450 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 --2- 172.21.15.50:0/3549617154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41040d0cc0 0x7f4104155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T21:21:42.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 >> 172.21.15.50:0/3549617154 conn(0x7f410409d5e0 msgr2=0x7f41040d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T21:21:42.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 shutdown_connections 2024-04-02T21:21:42.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T21:21:42.399+0000 7f410941b640 1 -- 172.21.15.50:0/3549617154 wait complete. 2024-04-02T21:21:42.413 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-02T21:21:42.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T21:21:42.413 DEBUG:teuthology.orchestra.run.smithi050:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-02T21:21:42.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T21:21:42.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T21:21:42.445 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T21:21:42.660 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T21:21:42.660 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T21:21:42.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057145,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 172.21.15.50:6789:/test /tmp/tmp.rLt4hGA3Uftest ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=057e6c06-7c98-48a0-9312-9e11e9abe7fe,acl 0 0 172.21.15.50:6789:/scratch /tmp/tmp.9wSzMl1nX5scratch ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=057e6c06-7c98-48a0-9312-9e11e9abe7fe,acl 0 0 2024-04-02T21:21:42.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T21:21:42.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T21:21:42.677 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-02T21:21:42.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T21:21:42.733 INFO:teuthology.orchestra.run.smithi050.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-02T21:21:42.733 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T21:21:42.733 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T21:21:42.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18592 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-02T21:21:42.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T21:21:42.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T21:21:42.760 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-02T21:21:42.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T21:21:42.831 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-02T21:21:42.831 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T21:21:42.831 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T21:21:42.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18592 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-02T21:21:42.888 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T21:21:42.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T21:21:42.888 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-02T21:21:42.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T21:21:42.943 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-02T21:21:44.453 INFO:tasks.ceph.osd.0:Sent signal 15 2024-04-02T21:21:44.454 INFO:tasks.ceph.osd.1:Sent signal 15 2024-04-02T21:21:44.454 INFO:tasks.ceph.osd.3:Sent signal 15 2024-04-02T21:21:44.454 INFO:tasks.ceph.osd.4:Sent signal 15 2024-04-02T21:21:44.455 INFO:tasks.ceph.osd.6:Sent signal 15 2024-04-02T21:21:44.455 INFO:tasks.ceph.osd.7:Sent signal 15 2024-04-02T21:21:44.455 INFO:tasks.ceph.mds.a:Sent signal 15 2024-04-02T21:21:44.455 INFO:tasks.ceph.mds.c:Sent signal 15 2024-04-02T21:21:44.455 INFO:tasks.ceph.mds.b:Sent signal 15 2024-04-02T21:21:44.456 INFO:tasks.ceph.mds.d:Sent signal 15 2024-04-02T21:21:44.456 INFO:tasks.ceph.mon.a:Sent signal 15 2024-04-02T21:21:44.456 INFO:tasks.ceph.mon.b:Sent signal 15 2024-04-02T21:21:44.456 INFO:tasks.ceph.mon.c:Sent signal 15 2024-04-02T21:21:44.456 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-04-02T21:21:44.457 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-04-02T21:21:44.458 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T21:21:44.456+0000 7fd244e01640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37715) UID: 0 2024-04-02T21:21:44.458 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T21:21:44.456+0000 7fd244e01640 -1 osd.4 45 *** Got signal Terminated *** 2024-04-02T21:21:44.458 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-02T21:21:44.456+0000 7fd244e01640 -1 osd.4 45 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T21:21:44.458 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T21:21:44.456+0000 7f1ff3801640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 37799) UID: 0 2024-04-02T21:21:44.458 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T21:21:44.456+0000 7f1ff3801640 -1 osd.0 45 *** Got signal Terminated *** 2024-04-02T21:21:44.458 INFO:tasks.ceph.osd.0.smithi050.stderr:2024-04-02T21:21:44.456+0000 7f1ff3801640 -1 osd.0 45 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T21:21:44.459 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T21:21:44.457+0000 7f1da7c55640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 37794) UID: 0 2024-04-02T21:21:44.459 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T21:21:44.457+0000 7f1da7c55640 -1 osd.1 45 *** Got signal Terminated *** 2024-04-02T21:21:44.459 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-04-02T21:21:44.457+0000 7f1da7c55640 -1 osd.1 45 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T21:21:44.459 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T21:21:44.457+0000 7f0530f8c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37717) UID: 0 2024-04-02T21:21:44.459 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T21:21:44.457+0000 7f0530f8c640 -1 osd.6 45 *** Got signal Terminated *** 2024-04-02T21:21:44.459 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-02T21:21:44.457+0000 7f0530f8c640 -1 osd.6 45 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T21:21:44.460 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T21:21:44.457+0000 7f2262b9a640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 37807) UID: 0 2024-04-02T21:21:44.460 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T21:21:44.457+0000 7f2262b9a640 -1 osd.3 45 *** Got signal Terminated *** 2024-04-02T21:21:44.460 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-04-02T21:21:44.457+0000 7f2262b9a640 -1 osd.3 45 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T21:21:44.461 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T21:21:44.458+0000 7fe34ca6b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 51310) UID: 0 2024-04-02T21:21:44.461 INFO:tasks.ceph.mds.a.smithi050.stderr:2024-04-02T21:21:44.458+0000 7fe34ca6b640 -1 mds.a *** got signal Terminated *** 2024-04-02T21:21:44.461 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T21:21:44.457+0000 7ff98796e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37720) UID: 0 2024-04-02T21:21:44.461 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T21:21:44.457+0000 7ff98796e640 -1 osd.7 45 *** Got signal Terminated *** 2024-04-02T21:21:44.462 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-02T21:21:44.457+0000 7ff98796e640 -1 osd.7 45 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T21:21:44.462 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T21:21:44.458+0000 7fcf2c643640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 51311) UID: 0 2024-04-02T21:21:44.462 INFO:tasks.ceph.mds.c.smithi050.stderr:2024-04-02T21:21:44.458+0000 7fcf2c643640 -1 mds.c *** got signal Terminated *** 2024-04-02T21:21:44.462 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-04-02T21:21:44.458+0000 7f05caef9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 44374) UID: 0 2024-04-02T21:21:44.462 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-04-02T21:21:44.458+0000 7f05caef9640 -1 mds.b *** got signal Terminated *** 2024-04-02T21:21:44.463 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-04-02T21:21:44.459+0000 7f36de6e1640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 37521) UID: 0 2024-04-02T21:21:44.463 INFO:tasks.ceph.mon.a.smithi050.stderr:2024-04-02T21:21:44.459+0000 7f36de6e1640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T21:21:44.463 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-02T21:21:44.458+0000 7f35a7679640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 44375) UID: 0 2024-04-02T21:21:44.463 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-02T21:21:44.458+0000 7f35a7679640 -1 mds.d *** got signal Terminated *** 2024-04-02T21:21:44.463 INFO:tasks.ceph.mon.c.smithi188.stderr:2024-04-02T21:21:44.459+0000 7f47348e2640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 37148) UID: 0 2024-04-02T21:21:44.463 INFO:tasks.ceph.mon.c.smithi188.stderr:2024-04-02T21:21:44.459+0000 7f47348e2640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-04-02T21:21:44.464 INFO:tasks.ceph.mon.b.smithi188.stderr:2024-04-02T21:21:44.459+0000 7fab5081c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 37146) UID: 0 2024-04-02T21:21:44.464 INFO:tasks.ceph.mon.b.smithi188.stderr:2024-04-02T21:21:44.459+0000 7fab5081c640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-04-02T21:21:44.658 INFO:tasks.ceph.mgr.y.smithi050.stderr:daemon-helper: command crashed with signal 15 2024-04-02T21:23:40.229 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:23:40.233 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:23:40.277 INFO:teuthology.orchestra.run.smithi188.stderr:ceph-osd: no process found 2024-04-02T21:23:40.295 INFO:teuthology.orchestra.run.smithi050.stderr:ceph-osd: no process found 2024-04-02T21:26:24.760 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:26:24.763 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:26:55.091 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:26:55.095 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:27:25.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:27:25.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:27:55.287 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:27:55.291 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:28:25.414 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:28:25.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:28:55.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:28:55.491 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:29:25.618 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:29:25.622 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:29:55.690 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:29:55.693 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:30:25.840 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:30:25.844 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:30:55.976 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:30:55.980 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:31:26.086 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:31:26.090 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:31:56.214 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:31:56.218 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:32:26.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:32:26.289 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:32:56.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:32:56.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:33:26.513 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:33:26.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:33:56.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:33:56.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:34:26.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:34:26.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:34:56.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:34:56.889 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:35:26.997 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:35:27.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:35:57.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:35:57.111 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:36:27.243 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:36:27.247 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:36:57.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:36:57.358 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:37:27.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:37:27.485 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:37:57.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:37:57.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:38:27.686 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:38:27.689 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:38:57.755 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:38:57.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:39:27.899 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:39:27.903 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:39:57.986 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:39:57.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:40:28.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:40:28.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:40:58.290 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:40:58.294 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:41:28.418 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:41:28.422 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:41:58.536 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:41:58.539 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:42:28.605 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:42:28.609 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:42:58.741 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:42:58.745 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:43:28.918 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:43:28.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:43:59.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:43:59.041 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:44:29.104 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:44:29.107 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:44:59.251 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:44:59.255 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:45:29.361 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:45:29.365 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:45:59.505 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:45:59.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:46:29.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:46:29.580 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:46:59.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:46:59.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:47:29.853 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:47:29.857 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:47:59.926 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:47:59.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:48:30.082 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:48:30.088 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:49:00.154 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:49:00.158 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:49:30.288 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:49:30.291 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:50:00.383 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:50:00.386 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:50:30.518 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:50:30.522 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:51:00.677 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:51:00.681 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:51:30.813 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:51:30.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:52:00.950 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:52:00.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:52:31.101 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:52:31.105 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:53:01.175 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:53:01.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:53:31.247 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:53:31.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:54:01.383 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:54:01.387 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:54:31.453 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:54:31.456 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:55:01.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:55:01.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:55:31.681 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:55:31.685 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:56:01.817 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:56:01.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:56:31.888 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:56:31.891 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:57:02.005 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:57:02.012 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:57:32.174 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:57:32.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:58:02.393 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:58:02.396 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:58:32.503 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:58:32.506 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:59:02.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:59:02.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:59:32.724 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T21:59:32.728 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:00:02.818 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:00:02.823 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:00:32.962 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:00:32.965 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:01:03.038 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:01:03.042 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:01:33.175 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:01:33.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:02:03.294 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:02:03.298 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:02:33.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:02:33.466 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:03:03.599 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:03:03.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:03:33.668 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:03:33.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:04:03.795 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:04:03.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:04:33.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:04:33.888 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:05:04.041 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:05:04.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:05:34.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:05:34.147 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:06:04.279 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:06:04.282 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:06:34.462 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:06:34.466 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:07:04.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:07:04.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:07:34.677 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:07:34.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:08:04.812 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:08:04.816 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:08:34.881 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:08:34.885 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:09:04.952 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:09:04.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:09:35.077 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:09:35.081 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:10:05.172 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:10:05.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:10:35.299 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:10:35.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:11:05.367 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:11:05.370 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:11:35.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:11:35.515 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:12:05.681 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:12:05.685 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:12:35.865 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:12:35.868 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:13:05.938 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:13:05.942 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:13:36.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:13:36.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:14:06.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:14:06.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:14:36.350 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:14:36.354 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:15:06.519 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:15:06.523 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:15:36.588 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:15:36.592 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:16:06.656 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:16:06.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:16:36.728 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:16:36.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:17:06.844 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:17:06.847 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:17:37.040 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:17:37.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:18:07.111 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:18:07.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:18:37.268 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:18:37.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:19:07.404 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:19:07.407 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:19:37.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:19:37.544 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:20:07.634 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:20:07.637 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:20:37.744 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:20:37.748 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:21:07.821 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:21:07.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:21:37.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:21:37.964 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:22:08.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:22:08.037 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:22:38.106 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:22:38.109 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:23:08.314 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:23:08.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:23:38.441 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:23:38.444 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:24:08.587 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:24:08.591 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:24:38.853 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:24:38.857 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:25:09.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:25:09.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:25:39.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:25:39.454 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:26:09.602 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:26:09.605 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:26:39.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:26:39.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:27:09.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:27:09.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:27:39.825 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:27:39.829 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:28:09.977 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:28:09.980 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:28:40.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:28:40.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:29:10.265 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:29:10.269 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:29:40.449 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:29:40.453 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:30:10.587 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:30:10.590 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:30:40.656 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:30:40.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:31:10.883 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:31:10.887 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:31:40.948 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:31:40.951 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:32:11.037 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:32:11.041 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:32:41.164 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:32:41.168 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:33:11.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:33:11.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:33:41.369 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:33:41.373 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:34:11.438 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:34:11.441 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:34:41.599 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:34:41.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:35:11.671 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:35:11.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:35:41.870 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:35:41.873 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:36:12.131 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:36:12.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:36:42.377 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:36:42.381 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:37:12.517 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:37:12.521 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:37:42.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:37:42.594 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:38:12.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:38:12.708 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:38:42.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:38:42.810 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:39:12.959 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:39:12.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:39:43.186 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:39:43.190 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:40:13.252 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:40:13.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:40:43.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:40:43.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:41:13.535 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:41:13.538 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:41:43.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:41:43.615 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:42:13.839 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:42:13.843 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:42:43.992 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:42:43.996 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:43:14.344 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:43:14.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:43:44.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:43:44.475 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:44:14.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:44:14.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:44:44.725 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:44:44.729 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:45:14.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:45:14.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:45:44.913 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:45:44.916 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:46:14.989 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:46:14.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:46:45.142 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:46:45.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:47:15.302 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:47:15.306 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:47:45.369 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:47:45.372 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:48:15.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:48:15.486 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:48:45.835 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:48:45.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:49:15.995 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:49:15.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:49:46.323 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:49:46.326 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:50:16.466 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:50:16.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:50:46.536 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:50:46.539 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:51:16.671 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:51:16.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:51:46.740 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:51:46.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:52:16.835 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:52:16.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:52:46.993 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:52:46.996 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:53:17.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:53:17.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:53:47.390 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:53:47.394 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:54:17.567 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:54:17.570 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:54:47.637 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:54:47.643 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:55:17.714 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:55:17.717 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:55:47.832 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:55:47.836 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:56:17.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:56:17.907 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:56:48.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:56:48.032 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:57:18.123 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:57:18.126 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:57:48.258 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:57:48.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:58:18.456 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:58:18.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:58:48.613 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:58:48.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:59:18.687 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:59:18.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:59:48.867 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T22:59:48.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:00:19.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:00:19.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:00:49.296 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:00:49.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:01:19.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:01:19.380 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:01:49.609 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:01:49.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:02:19.678 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:02:19.682 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:02:49.780 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:02:49.784 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:03:19.925 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:03:19.929 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:03:50.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:03:50.015 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:04:20.170 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:04:20.173 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:04:50.352 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:04:50.356 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:05:20.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:05:20.512 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:05:50.586 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:05:50.589 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:06:20.722 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:06:20.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:06:50.796 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:06:50.800 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:07:20.868 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:07:20.872 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:07:51.161 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:07:51.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:08:21.305 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:08:21.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:08:51.457 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:08:51.461 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:09:21.793 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:09:21.797 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:09:51.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:09:51.890 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:10:22.072 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:10:22.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:10:52.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:10:52.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:11:22.202 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:11:22.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:11:52.277 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:11:52.280 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:12:22.382 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:12:22.385 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:12:52.509 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:12:52.512 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:13:22.584 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:13:22.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:13:52.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:13:52.709 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:14:22.783 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:14:22.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:14:52.905 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:14:52.909 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:15:23.274 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:15:23.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:15:53.392 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:15:53.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:16:23.527 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:16:23.530 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:16:53.637 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:16:53.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:17:23.704 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:17:23.708 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:17:54.015 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:17:54.019 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:18:24.150 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:18:24.153 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:18:54.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:18:54.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:19:24.595 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:19:24.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:19:54.663 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:19:54.666 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:20:24.815 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:20:24.819 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:20:54.883 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:20:54.886 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:21:25.002 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:21:25.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:21:55.095 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:21:55.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:22:25.222 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:22:25.226 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:22:55.399 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:22:55.402 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:23:25.534 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:23:25.538 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:23:55.610 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:23:55.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:24:26.004 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:24:26.008 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:24:56.122 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:24:56.126 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:25:26.367 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:25:26.370 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:25:56.501 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:25:56.504 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:26:26.569 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:26:26.573 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:26:56.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:26:56.742 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:27:26.815 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:27:26.818 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:27:56.976 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:27:56.980 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:28:27.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:28:27.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:28:57.205 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:28:57.209 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:29:27.348 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:29:27.352 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:29:57.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:29:57.487 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:30:27.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:30:27.615 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:30:57.713 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:30:57.716 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:31:27.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:31:27.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:31:57.876 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:31:57.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:32:28.020 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:32:28.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:32:58.172 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:32:58.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:33:28.307 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:33:28.310 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:33:58.425 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:33:58.429 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:34:28.602 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:34:28.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:34:58.738 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:34:58.742 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:35:28.865 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:35:28.869 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:35:58.991 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:35:58.994 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:36:29.110 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:36:29.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:36:59.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:36:59.257 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:37:29.405 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:37:29.408 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:37:59.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:37:59.544 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:38:29.683 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:38:29.686 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:38:59.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:38:59.823 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:39:29.986 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:39:29.990 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:40:00.121 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:40:00.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:40:30.240 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:40:30.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:41:00.368 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:41:00.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:41:30.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:41:30.489 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:42:00.621 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:42:00.625 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:42:30.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:42:30.742 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:43:00.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:43:00.878 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:43:31.000 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:43:31.004 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:44:01.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:44:01.149 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:44:31.289 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:44:31.292 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:45:01.415 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:45:01.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:45:31.559 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:45:31.563 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:46:01.711 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:46:01.715 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:46:31.838 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:46:31.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:47:01.975 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:47:01.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:47:32.113 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:47:32.116 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:48:02.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:48:02.186 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:48:32.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:48:32.331 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:49:02.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:49:02.484 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:49:32.601 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:49:32.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:50:02.729 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:50:02.733 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:50:32.865 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:50:32.869 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:51:03.002 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:51:03.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:51:33.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:51:33.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:52:03.275 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:52:03.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:52:33.420 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:52:33.423 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:53:03.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:53:03.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:53:33.717 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:53:33.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:54:03.853 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:54:03.857 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:54:33.998 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:54:34.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:55:04.126 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:55:04.129 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:55:34.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:55:34.257 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:56:04.395 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:56:04.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:56:34.517 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:56:34.520 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:57:04.644 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:57:04.647 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:57:34.755 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:57:34.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:58:04.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:58:04.879 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:58:35.010 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:58:35.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:59:05.146 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:59:05.150 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:59:35.274 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T23:59:35.771 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:00:05.902 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:00:05.905 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:00:36.071 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:00:36.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:01:06.199 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:01:06.203 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:01:36.320 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:01:36.324 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:02:06.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:02:06.467 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:02:36.608 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:02:36.612 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:03:06.770 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:03:06.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:03:36.906 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:03:36.910 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:04:07.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:04:07.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:04:37.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:04:37.187 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:05:07.319 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:05:07.323 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:05:37.446 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:05:37.450 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:06:07.582 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:06:07.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:06:37.718 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:06:37.722 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:07:07.845 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:07:07.848 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:07:37.980 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:07:37.985 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:08:08.124 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:08:08.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:08:38.260 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:08:38.263 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:09:08.379 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:09:08.383 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:09:38.522 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:09:38.526 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:10:08.642 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:10:08.646 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:10:38.795 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:10:38.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:11:08.921 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:11:08.925 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:11:39.049 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:11:39.053 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:12:09.201 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:12:09.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:12:39.345 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:12:39.348 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:13:09.522 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:13:09.525 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:13:39.666 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:13:39.670 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:14:09.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:14:09.805 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:14:39.937 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:14:39.941 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:15:10.072 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:15:10.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:15:40.216 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:15:40.220 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:16:10.369 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:16:10.373 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:16:40.498 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:16:40.502 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:17:10.631 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:17:10.634 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:17:40.708 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:17:40.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:18:10.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:18:10.863 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:18:40.996 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:18:40.999 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:19:11.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:19:11.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:19:41.276 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:19:41.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:20:11.394 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:20:11.397 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:20:41.545 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:20:41.549 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:21:11.689 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:21:11.692 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:21:41.824 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:21:41.828 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:22:11.942 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:22:11.946 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:22:42.085 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:22:42.088 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:23:12.213 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:23:12.217 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:23:42.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:23:42.358 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:24:12.474 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:24:12.477 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:24:42.633 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:24:42.636 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:25:12.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:25:12.772 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:25:42.912 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:25:42.917 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:26:13.047 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:26:13.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:26:43.183 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:26:43.189 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:27:13.308 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:27:13.312 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:27:43.461 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:27:43.465 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:28:13.604 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:28:13.608 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:28:43.764 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:28:43.768 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:29:13.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:29:13.919 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:29:44.060 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:29:44.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:30:14.195 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:30:14.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:30:44.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:30:44.359 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:31:14.491 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:31:14.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:31:44.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:31:44.629 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:32:14.753 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:32:14.757 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:32:44.889 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:32:44.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:33:14.998 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:33:15.002 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:33:45.142 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:33:45.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:34:15.326 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:34:15.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:34:45.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:34:45.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:35:15.606 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:35:15.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:35:45.734 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:35:45.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:36:15.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:36:15.863 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:36:45.978 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:36:45.982 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:37:16.122 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:37:16.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:37:46.257 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:37:46.260 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:38:16.384 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:38:16.388 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:38:46.503 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:38:46.507 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:39:16.638 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:39:16.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:39:46.757 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:39:46.761 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:40:16.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:40:16.878 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:40:47.019 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:40:47.023 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:41:17.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:41:17.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:41:47.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:41:47.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:42:17.417 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:42:17.421 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:42:47.535 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:42:47.539 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:43:17.671 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:43:17.675 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:43:47.806 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:43:47.810 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:44:17.967 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:44:17.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:44:48.102 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:44:48.106 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:45:18.229 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:45:18.233 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:45:48.348 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:45:48.351 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:46:18.474 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:46:18.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:46:48.606 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:46:48.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:47:18.720 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:47:18.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:47:48.798 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:47:48.801 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:48:18.958 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:48:18.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:48:49.110 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:48:49.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:49:19.253 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:49:19.257 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:49:49.380 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:49:49.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:50:19.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:50:19.511 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:50:49.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:50:49.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:51:19.796 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:51:19.799 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:51:49.956 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:51:49.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:52:20.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:52:20.132 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:52:50.260 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:52:50.263 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:53:20.387 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:53:20.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:53:50.506 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:53:50.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:54:20.650 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:54:20.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:54:50.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:54:50.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:55:20.953 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:55:20.957 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:55:51.079 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:55:51.083 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:56:21.198 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:56:21.202 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:56:51.349 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:56:51.352 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:57:21.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:57:21.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:57:51.619 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:57:51.623 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:58:21.771 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:58:21.774 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:58:51.907 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:58:51.911 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:59:22.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:59:22.046 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:59:52.169 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T00:59:52.172 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:00:22.296 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:00:22.299 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:00:52.434 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:00:52.438 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:01:22.558 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:01:22.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:01:52.685 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:01:52.689 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:02:22.837 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:02:22.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:02:52.998 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:02:53.002 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:03:23.133 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:03:23.136 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:03:53.284 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:03:53.288 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:04:23.436 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:04:23.439 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:04:53.572 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:04:53.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:05:23.740 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:05:23.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:05:53.876 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:05:53.879 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:06:24.003 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:06:24.006 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:06:54.120 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:06:54.123 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:07:24.247 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:07:24.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:07:54.416 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:07:54.419 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:08:24.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:08:24.556 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:08:54.687 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:08:54.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:09:24.847 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:09:24.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:09:54.983 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:09:56.974 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:10:27.059 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:10:27.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:10:57.189 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:10:57.193 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:11:27.349 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:11:27.352 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:11:57.490 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:11:57.493 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:12:27.600 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:12:27.603 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:12:57.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:12:57.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:13:27.904 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:13:27.907 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:13:58.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:13:58.059 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:14:28.175 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:14:28.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:14:58.310 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:14:58.314 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:15:28.470 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:15:28.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:15:58.597 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:15:58.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:16:28.733 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:16:28.736 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:16:58.943 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:16:58.946 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:17:29.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:17:29.015 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:17:59.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:17:59.109 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:18:29.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:18:29.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:18:59.368 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:18:59.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:19:29.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:19:29.491 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:19:59.605 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:19:59.609 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:20:29.766 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:20:29.769 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:20:59.893 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:20:59.897 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:21:30.011 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:21:30.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:22:00.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:22:00.167 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:22:30.307 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:22:30.311 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:23:00.442 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:23:00.446 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:23:30.594 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:23:30.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:24:00.721 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:24:00.724 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:24:30.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:24:30.877 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:25:01.017 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:25:01.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:25:31.160 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:25:31.163 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:26:01.296 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:26:01.299 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:26:31.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:26:31.452 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:27:01.583 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:27:01.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:27:31.744 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:27:31.748 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:28:01.870 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:28:01.874 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:28:32.006 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:28:32.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:29:02.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:29:02.144 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:29:32.276 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:29:32.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:30:02.402 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:30:02.405 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:30:32.537 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:30:32.541 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:31:02.608 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:31:02.611 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:31:32.734 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:31:32.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:32:02.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:32:02.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:32:33.012 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:32:33.016 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:33:03.156 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:33:03.159 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:33:33.282 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:33:33.286 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:34:03.443 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:34:03.447 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:34:33.571 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:34:33.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:35:03.687 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:35:03.692 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:35:33.806 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:35:33.809 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:36:03.941 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:36:03.945 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:36:34.068 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:36:34.073 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:37:04.205 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:37:04.208 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:37:34.347 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:37:34.350 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:38:04.465 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:38:04.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:38:34.618 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:38:34.622 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:39:04.778 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:39:04.782 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:39:34.939 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:39:34.942 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:40:05.082 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:40:05.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:40:35.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:40:35.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:41:05.353 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:41:05.357 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:41:35.480 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:41:35.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:42:05.598 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:42:05.602 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:42:35.751 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:42:35.754 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:43:05.887 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:43:05.890 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:43:36.047 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:43:36.050 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:44:06.198 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:44:06.201 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:44:36.359 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:44:36.362 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:45:06.506 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:45:06.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:45:36.646 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:45:36.650 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:46:06.791 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:46:06.794 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:46:36.908 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:46:36.912 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:47:07.044 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:47:07.047 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:47:37.162 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:47:37.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:48:07.298 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:48:07.301 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:48:37.416 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:48:37.420 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:49:07.561 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:49:07.564 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:49:37.712 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:49:37.715 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:50:07.840 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:50:07.843 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:50:37.992 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:50:37.995 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:51:08.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:51:08.130 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:51:38.279 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:51:38.282 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:52:08.406 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:52:08.409 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:52:38.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:52:38.570 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:53:08.677 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:53:08.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:53:38.804 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:53:38.809 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:54:08.956 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:54:08.959 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:54:39.108 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:54:39.112 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:55:09.268 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:55:09.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:55:39.420 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:55:39.424 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:56:09.540 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:56:09.543 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:56:39.616 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:56:39.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:57:09.751 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:57:09.754 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:57:39.928 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:57:39.932 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:58:10.089 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:58:10.093 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:58:40.224 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:58:40.228 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:59:10.368 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:59:10.372 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:59:40.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T01:59:40.515 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:00:10.664 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:00:10.667 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:00:40.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:00:40.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:01:10.934 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:01:10.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:01:41.070 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:01:41.073 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:02:11.196 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:02:11.201 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:02:41.357 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:02:41.361 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:03:11.492 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:03:11.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:03:41.644 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:03:41.648 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:04:11.779 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:04:11.782 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:04:41.923 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:04:41.926 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:05:12.083 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:05:12.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:05:42.210 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:05:42.213 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:06:12.370 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:06:12.374 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:06:42.523 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:06:42.527 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:07:12.665 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:07:12.669 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:07:42.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:07:42.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:08:12.953 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:08:12.956 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:08:43.097 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:08:43.101 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:09:13.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:09:13.244 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:09:43.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:09:43.379 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:10:13.553 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:10:13.557 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:10:43.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:10:43.709 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:11:13.781 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:11:13.784 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:11:43.925 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:11:43.929 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:12:14.069 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:12:14.073 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:12:44.221 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:12:44.225 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:13:14.347 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:13:14.351 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:13:44.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:13:44.487 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:14:14.618 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:14:14.621 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:14:44.754 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:14:44.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:15:14.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:15:14.901 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:15:45.050 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:15:45.053 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:16:15.202 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:16:15.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:16:45.320 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:16:45.323 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:17:15.464 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:17:15.467 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:17:45.532 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:17:45.535 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:18:15.659 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:18:15.663 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:18:45.786 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:18:45.791 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:19:15.930 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:19:15.934 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:19:46.065 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:19:46.068 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:20:16.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:20:16.187 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:20:46.303 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:20:46.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:21:16.421 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:21:16.424 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:21:46.575 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:21:46.578 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:22:16.734 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:22:16.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:22:46.886 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:22:46.889 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:23:17.039 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:23:17.042 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:23:47.173 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:23:47.176 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:24:17.292 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:24:17.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:24:47.436 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:24:47.439 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:25:17.570 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:25:17.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:25:47.714 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:25:47.718 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:26:17.833 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:26:17.836 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:26:47.968 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:26:47.972 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:27:18.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:27:18.107 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:27:48.231 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:27:48.234 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:28:18.350 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:28:18.354 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:28:48.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:28:48.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:29:18.620 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:29:18.623 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:29:48.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:29:48.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:30:18.891 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:30:18.895 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:30:49.043 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:30:49.046 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:31:19.178 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:31:19.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:31:49.330 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:31:49.334 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:32:19.474 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:32:19.477 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:32:49.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:32:49.629 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:33:19.786 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:33:19.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:33:49.940 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:33:49.943 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:34:20.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:34:20.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:34:50.201 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:34:50.204 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:35:20.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:35:20.331 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:35:50.480 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:35:50.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:36:20.631 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:36:20.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:36:50.784 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:36:50.788 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:37:20.919 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:37:20.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:37:51.071 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:37:51.074 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:38:21.206 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:38:21.209 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:38:51.341 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:38:51.344 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:39:21.460 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:39:21.464 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:39:51.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:39:51.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:40:21.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:40:21.760 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:40:51.896 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:40:51.899 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:41:22.026 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:41:22.029 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:41:52.170 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:41:52.174 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:42:22.324 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:42:22.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:42:52.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:42:52.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:43:22.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:43:22.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:43:52.755 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:43:52.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:44:22.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:44:22.919 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:44:53.059 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:44:53.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:45:23.211 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:45:23.214 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:45:53.362 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:45:53.366 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:46:23.506 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:46:23.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:46:53.650 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:46:53.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:47:23.835 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:47:23.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:47:53.920 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:47:53.924 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:48:24.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:48:24.058 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:48:54.190 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:48:54.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:49:24.351 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:49:24.354 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:49:54.512 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:49:54.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:50:24.663 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:50:24.666 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:50:54.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:50:54.811 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:51:24.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:51:24.963 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:51:55.086 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:51:55.089 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:52:25.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:52:25.258 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:52:55.390 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:52:55.393 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:53:25.542 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:53:25.545 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:53:55.660 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:53:55.663 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:54:25.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:54:25.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:54:55.931 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:54:55.935 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:55:26.074 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:55:26.077 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:55:56.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:55:56.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:56:26.353 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:56:26.357 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:56:56.505 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:56:56.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:57:26.674 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:57:26.678 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:57:56.826 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:57:56.830 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:58:26.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:58:26.965 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:58:57.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:58:57.108 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:59:27.265 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:59:27.269 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:59:57.442 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T02:59:57.446 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:00:27.569 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:00:27.572 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:00:57.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:00:57.719 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:01:27.848 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:01:27.852 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:01:57.974 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:01:57.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:02:28.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:02:28.131 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:02:58.262 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:02:58.265 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:03:28.406 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:03:28.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:03:58.532 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:03:58.535 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:04:28.652 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:04:28.655 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:04:58.786 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:04:58.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:05:28.922 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:05:28.925 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:05:59.073 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:05:59.077 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:06:29.200 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:06:29.203 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:06:59.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:06:59.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:07:29.463 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:07:29.467 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:07:59.615 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:07:59.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:08:29.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:08:29.770 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:08:59.918 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:08:59.921 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:09:30.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:09:30.067 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:10:00.190 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:10:00.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:10:30.325 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:10:30.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:11:00.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:11:00.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:11:30.603 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:11:30.607 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:12:00.747 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:12:00.751 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:12:30.899 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:12:30.902 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:13:01.051 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:13:01.055 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:13:31.179 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:13:31.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:14:01.305 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:14:01.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:14:31.449 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:14:31.452 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:15:01.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:15:01.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:15:31.702 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:15:31.706 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:16:01.847 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:16:01.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:16:31.981 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:16:31.985 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:17:02.117 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:17:02.120 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:17:32.236 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:17:32.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:18:02.313 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:18:02.316 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:18:32.440 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:18:32.443 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:19:02.591 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:19:02.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:19:32.719 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:19:32.722 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:20:02.854 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:20:02.857 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:20:32.997 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:20:33.001 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:21:03.080 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:21:03.083 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:21:33.228 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:21:33.233 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:22:03.362 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:22:03.365 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:22:33.480 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:22:33.484 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:23:03.640 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:23:03.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:23:33.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:23:33.771 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:24:03.923 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:24:03.926 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:24:34.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:24:34.058 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:25:04.173 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:25:04.177 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:25:34.326 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:25:34.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:26:04.472 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:26:04.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:26:34.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:26:34.616 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:27:04.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:27:04.760 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:27:34.908 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:27:34.912 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:28:05.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:28:05.038 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:28:35.170 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:28:35.174 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:29:05.314 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:29:05.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:29:35.465 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:29:35.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:30:05.601 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:30:05.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:30:35.753 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:30:35.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:31:05.889 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:31:05.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:31:35.999 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:31:36.003 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:32:06.168 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:32:06.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:32:36.294 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:32:36.298 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:33:06.413 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:33:06.419 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:33:36.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:33:36.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:34:06.725 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:34:06.729 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:34:36.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:34:36.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:35:06.946 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:35:06.950 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:35:37.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:35:37.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:36:07.192 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:36:07.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:36:37.344 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:36:37.348 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:37:07.529 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:37:07.533 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:37:37.672 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:37:37.677 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:38:07.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:38:07.803 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:38:37.951 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:38:37.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:39:08.078 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:39:08.081 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:39:38.247 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:39:38.251 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:40:08.399 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:40:08.403 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:40:38.535 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:40:38.538 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:41:08.686 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:41:08.690 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:41:38.830 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:41:38.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:42:08.974 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:42:08.977 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:42:39.108 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:42:39.112 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:43:09.235 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:43:09.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:43:39.387 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:43:39.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:44:09.523 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:44:09.527 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:44:39.683 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:44:39.687 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:45:09.818 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:45:09.822 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:45:39.953 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:45:39.956 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:46:10.095 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:46:10.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:46:40.236 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:46:40.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:47:10.362 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:47:10.366 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:47:40.440 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:47:40.444 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:48:10.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:48:10.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:48:40.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:48:40.719 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:49:10.859 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:49:10.862 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:49:41.026 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:49:41.030 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:50:11.169 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:50:11.172 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:50:41.329 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:50:41.333 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:51:11.455 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:51:11.458 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:51:41.581 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:51:41.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:52:11.707 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:52:11.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:52:41.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:52:41.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:53:11.968 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:53:11.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:53:42.094 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:53:42.098 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:54:12.246 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:54:12.250 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:54:42.364 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:54:42.369 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:55:12.515 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:55:12.519 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:55:42.667 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:55:42.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:56:12.826 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:56:12.829 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:56:42.970 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:56:42.974 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:57:13.096 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:57:13.100 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:57:43.223 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:57:43.227 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:58:13.341 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:58:13.345 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:58:43.493 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:58:43.496 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:59:13.620 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:59:13.623 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:59:43.732 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T03:59:43.735 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:00:13.856 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:00:13.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:00:43.999 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:00:44.002 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:01:14.142 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:01:14.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:01:44.295 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:01:44.299 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:02:14.429 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:02:14.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:02:44.555 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:02:44.558 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:03:14.691 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:03:14.695 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:03:44.842 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:03:44.846 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:04:14.978 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:04:14.982 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:04:45.105 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:04:45.109 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:05:15.264 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:05:15.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:05:45.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:05:45.377 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:06:15.534 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:06:15.538 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:06:45.669 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:06:45.672 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:07:15.821 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:07:15.825 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:07:45.947 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:07:45.950 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:08:16.075 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:08:16.078 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:08:46.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:08:46.221 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:09:16.370 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:09:16.373 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:09:46.496 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:09:46.499 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:10:16.632 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:10:16.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:10:46.783 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:10:46.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:11:16.910 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:11:16.913 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:11:47.045 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:11:47.048 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:12:17.189 DEBUG:teuthology.orchestra.run.smithi050:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:12:17.193 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-03T04:12:17.973 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-04-03T04:12:17.973 DEBUG:teuthology.task.console_log:Killing console logger for smithi050 2024-04-03T04:12:17.974 DEBUG:teuthology.task.console_log:Killing console logger for smithi188 2024-04-03T04:12:17.974 DEBUG:teuthology.exit:Finished running handlers